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Executive  Summary 


Concept  engineering,  as  addressed  in  this  report,  is  the  act  of  translating  stakeholder 
needs  into  a  operational  concept  (or  CONOPS)  for  a  product  or  system  capable  of 
satisfying  the  goals  set  forth  by  the  stakeholder.  This  report  summarizes  the  work 
performed  in  the  second  phase  of  research  on  development  of  an  approach  to  use 
graphical  and  collaborative  approaches  to  generating  an  operational  concept  for  a  new 
system. 

The  goal  of  this  research  phase  was  to  develop  an  initial  set  of  primitives  (or  core  terms) 
organized  into  a  hierarchy  (taxonomy)  that  is  reusable  when  creating  a  collection  of 
future  scenarios  for  different  domain.  The  initial  approach  taken  was  to  decompose  a 
well  tested  scenario,  the  Noncombatant  Evacuation  Operation  Intelligence  Gathering 
Scenario  (NEO).  The  first  step  was  to  perform  a  cognitive  task  analysis  of  what  was 
necessary  to  plan  the  scenario,  and  then  to  perform  a  decomposition  of  terms  and 
concepts  used  in  the  scenario.  This  effort  resulted  in  a  number  of  insights  being 
discovered,  which  advance  the  research  to  the  next  step.  However,  the  NEO  scenario 
was  found  wanting  in  complexity  and  information.  In  discussions  with  the  sponsor  of 
this  research,  it  was  recommended  the  team  investigate  a  news  agency  as  a  more 
representative  analog  for  this  research. 

When  a  number  of  new  agencies  were  investigated,  and  decomposed,  a  rich  taxonomy 
emerged  that  can  be  the  basis  for  new  domains.  While  each  new  domain  will  require  an 
expansion  of  the  taxonomy  with  new  terms  and  concepts,  many  are  common.  The 
notion  of  transporting  objects  between  locations,  information  gathering,  and 
communications  are  core  to  many  actions  that  might  be  modeled.  For  instance,  when 
looking  at  a  military  mission  of  close  air  support,  objects  are  moving  from  one  location 
to  another,  at  specific  times,  and  there  is  a  high  degree  of  communication  and 
collaboration  necessary.  The  same  can  be  said  for  emergency  response  for  an  oil  spill. 
While  the  objects  may  be  domain  specific,  the  movement,  communications, 
collaborations  and  actions  are  common  at  some  level  of  abstraction.  If  those  actions  can 
be  collected  and  represented  graphically,  then  the  definition  of  new  scenarios  can  be 
accomplished  quicker,  and  can  be  easier  to  understand. 

This  research  report  also  examines  an  easy  to  understand  graphical  modeling  approach 
of  snap  together  pieces  associated  with  the  developed  taxonomy  to  aid  the  creation  of 
scenarios,  or  operational  concepts. 

Finally,  this  report  provides  recommendations  for  follow-on  research  in  this  area,  and 
proposes  the  development  of  a  system  that  will  aid  the  concept  engineering  process  to 
produce  graphical  operations  concept  building  on  the  reusable  taxonomy  developed  in 
the  research  conducted  in  this  phase. 


Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 

UNCLASSIFIED 

3 


This  page  intentionally  left  blank 


Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 


UNCLASSIFIED 

4 


Table  of  Contents 


Executive  Summary . 3 

Table  of  Contents . 5 

Figures  and  Tables . 7 

1  Purpose . 9 

1.1  Problem  Statement . 9 

1.2  Objectives . 9 

1.3  Research  Approach . 10 

2  Introduction  to  Concept  Engineering . 11 

2.1  Definitions . 11 

3  Cognitive  Task  Analysis  and  the  NEO  Scenario . 12 

3.1  Literature  Review . 12 

3.2  Procedures . 14 

3.2.1  Cognitive  T ask  Analysis . 14 

3.3  Case  Studies  and  Results . 15 

3.3.1  NEO  Intelligence  Gathering  Task . 15 

3.3.2  NEO  Mission  Planning  Task . 17 

3.4  Recommendations  from  the  Study . 18 

3.4.1  Tool  Creation . 18 

3.4.2  Primitive  Creation . 18 

4  Primitives  and  Ontology  Research . 20 

4.1  Approach . 20 

4.2  Modeling  the  Ontology . 20 

4.2.1  Tool  Use . 20 

4.2.2  Modeling  Methodology . 27 

4.3  NEO  Scenario  /  Taxonomy . 27 

4.3.1  Methodology . 27 

4.3.2  NEO  Scenario  Ontology  Diagrams . 27 

4.4  News  Agency  Scenario  /  Taxonomy . 30 

4.4.1  Methodology . 30 

4.4.2  News  Agency  Scenario  Ontology  Diagrams . 31 

5  The  Scenario  Generator  Interface . 46 

5.1  Scenario  Introduction . 46 

5.1.1  New  Story . 46 

5.1.2  Corroborate  Source . 46 

5.1.3  New  Contact . 46 

5.1.4  Follow-up . 46 

Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 

UNCLASSIFIED 

5 


5.1. 5  Information  Synthesis . 47 

5.2  Scenario  Generator  Interface . 47 

5.2.1  Scenario  Generator . 47 

5.3  Applicability  to  Other  CONOPS/Domains . 54 

6  System  Architecture . 55 

6.1  High  Level  System  Architecture . 55 

6.2  Potential  CES  Components . 56 

6.2.1  Programming  Languages . 57 

6.2.2  Concept  Creation  Stations . 58 

6.2.3  Primitive  Creation . 59 

6.2.4  Scenario  Generator  and  Concept  Playback . 59 

7  Recommendations  for  Future  Research . 63 

7.1  Concept  Engineering  System  Development . 64 

7.1.1  Integration . 64 

7.1.2  CES  Development . 64 

7.2  Further  Scenario  Development . 65 

7.3  Experimentation . 66 

7.4  Proposed  Project  Plan . 66 

References . 68 

Appendix  A:  NEO  Intelligence  Gathering  Scenario . 71 

Appendix  B:  NEO  Mission  Planning  Scenario . 78 

Appendix  C:  News  Agency  Scenario  Primitives . 89 

Appendix  D:  NEO  Scenario  Ontology  Diagrams . 91 


Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 


UNCLASSIFIED 

6 


Figures  and  Tables 


Figure  l:  NEO  Intelligence  Gathering  Task  Concept  Map . 16 

Figure  2:  NEO  Mission  Planning  Task  Concept  Map . 17 

Figure  3:  NEO  Scenario  Protege  Diagram . 22 

Figure  4:  NEO  Scenario  SysML  Diagram . 22 

Figure  5:  NEO  Scenario  SysML  Package  Diagram . 24 

Figure  6:  Vehicle  Primitive  using  Protege . 26 

Figure  7:  Vehicle  Primitive  using  SparxEA . 26 

Figure  8:  NEO  Scenario  Top  Level  Primitives  Diagram . 29 

Figure  9:  News  Agency  Scenario  Ontology  Simplified  Object  Model . 31 

Figure  10:  News  Intelligence  Gathering  Primitives  Second  Level  Categories . 32 

Figure  11:  News  Medium  Primitive  Diagram . 33 

Figure  12:  Information  Primitive  Diagram . 35 

Figure  13:  Information:  Information  Types  Primitive  Diagram . 36 

Figure  14:  Information:  Information  Sources  Primitive  Diagram . 37 

Figure  15:  Information:  Information  Sources:  Contact  Primitive  Diagram . 38 

Figure  16:  News  Organization  Operations  Primitive  Diagram . 40 

Figure  17:  News  Organization  Operations:  Operational  Environment  Primitive  Diagram 
. 42 

Figure  18:  Transportation  Primitive  Diagram . 44 

Figure  19:  Personnel  Primitive  Diagram . 45 

Figure  20:  Scenario  Generator  Interface . 47 

Figure  21:  Scenario  Generator  Building  Blocks . 48 

Figure  22:  Scenario  1  -  New  News  Story . 50 

Figure  23:  Scenario  2  -  Corroborate  Source . 52 

Figure  24:  Recruit  New  Contact . 53 

Figure  25:  Concept  Engineering  System  (CES)  Logical  Architecture . 55 

Figure  26:  CES  Logical  Architecture  Connectivity . 56 

Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 


UNCLASSIFIED 

7 


Figure  27:  Activities  in  Phases  1  and  2 . 63 

Figure  28:  Inputs  to  Concept  Engineering  System . 64 

Figure  29:  Software  Scrum  Approach . 65 

Figure  30:  Process  for  Adding  Libraries . 65 

Figure  31:  Iterative  Approach . 66 

Figure  32:  NEO  Scenario  Noun  Primitives  from  Protege . 92 

Figure  33:  NEO  Scenario  Verb  Primitives  from  Protege . 93 

Figure  34:  NEO  Scenario  Adjective  Primitives  from  Protege . 94 

Figure  35:  NEO  Scenario  Top  Level  Primitives  from  Sparx . 95 

Figure  36:  NEO  Scenario  Equipment  Primitives  from  Sparx . 96 

Figure  37:  NEO  Scenario  Interface  Primitives  from  Sparx . 97 

Figure  38:  NEO  Scenario  Operations  Primitives  from  Sparx . 98 

Figure  39:  NEO  Scenario  Personnel  Primitives  from  Sparx . 99 

Figure  40:  NEO  Scenario  Risk  Assessment  Primitives  from  Sparx . too 

Figure  41:  NEO  Scenario  Infrastructure  Primitives  from  Sparx . 101 

Figure  42:  NEO  Scenario  Transportation  Primitives  from  Sparx . 102 

Figure  43:  NEO  Scenario  Environment  Primitives  from  Sparx . 103 

Table  1:  CTA  Data  Collection  with  Different  Levels  of  Researcher  Involvement . 13 

Table  2:  Example  Categories  of  Programming  Languages . 57 

Table  3:  Concept  Engineering  Research  Plan . 67 


Report  No.  SERC-2010-TR-007 
May  31,  2010 


UNCLASSIFIED 

8 


1  Purpose 


1.1  Problem  Statement 

The  weakest  link  in  systems  engineering  is  often  the  link  between  what  the  war-fighters 
or  analysts  needs,  and  what  the  development  team  “thinks”  they  need,  together  with  a 
shared  understanding  of  the  operational  environment  and  associated  constraints  and 
dependencies  [ASSE  2009].  The  concept  of  operations  can  be  a  means  to  bridge  this  gap 
significantly  [ANSI/AIAA 1992].  There  is  a  need  to  quickly  and  graphically  articulate  a 
concept  of  operations  (CONOPS)  for  new  missions,  business  processes,  and  feature  sets 
to  realize  a  shared  mental  model  and  understanding  of  the  mission,  and  potential 
solutions  across  a  set  of  diverse  stakeholders.  This  graphical  environment  has  the 
potential  to  become  a  mechanism  for  agile  stakeholder  expectation  elicitation.  It  is  likely 
that  this  environment  will  have  to  be  tailored  for  selected  application  domains  for 
operational,  mission,  and  semantic  consistency. 

A  systematic  and  disciplined  approach  should  be  undertaken  to  understand,  assess,  and 
catalogue  the  current  and  emerging  state  of  research  (elements  that  constitute  a  concept 
of  operations;  cognitive  sciences  and  mental  models;  and  modeling  tools  and 
environments)  to  allow  a  “domain  oriented”  environment  for  visualizing  a  mission 
concept  of  operations. 


1.2  Objectives 

This  phase  of  research  will  leverage  knowledge  gained  in  Phase  1  of  this  RT.  Specifically, 
these  7  tasks  will  provide  an  early  look  to  help  scope  and  further  define  the  broader 
tasks  in  Section  5.1  (Defining  the  Building  Blocks  for  a  Graphical  CONOP)  of  Phase  1 
Final  Technical  Report.  The  Phase  2  tasks  are: 

1)  Research,  identity  and  document  in  an  appropriate  manner,  an  initial  ontology 
for  use  in  constructing  a  graphical  CONOPS  toolbox  based  on  the  findings  from 
Phase  1.  Investigate  an  appropriate  tool  for  managing  and  defining  this  ontology 
to  allow  information  interchange  with  other  tools. 

2)  Ascertain  the  mental  model  content  most  appropriate  for  each  phase  of  the  agile 
CONOPS  process. 

3)  Create  an  initial  list  of  primitives  (and  the  necessary  information  for  each 
primitive)  to  ensure  reuse  based  on  the  identified  ontology 

4)  Identify  the  necessary  semantics  for  information  exchange  between  the  identified 
primitives 


Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 

UNCLASSIFIED 

9 


5)  Investigate  programming  languages/tools  appropriate  for  creating  a  graphical 
CONOPS 

6)  Modify  the  Non-Combatant  Evacuation  Operation  task  to  be  an  intelligence 
gathering  task  and  pilot  test  the  modifications.  This  will  include  a  small  sample  of 
participants  working  through  the  revised  task  to  ensure  the  modifications  are 
effective. 

7)  Identify  at  least  one  additional  intelligence  task  relevant  to  the  sponsor  for  use  in 
future  phases 

The  deliverable  from  Phase  2  will  be  a  final  report  documenting  the  results  of  the  phase, 
and  a  presentation  on  the  developed  ontology. 


1.3  Research  Approach 

The  research  was  approached  from  a  number  of  methods.  The  first  was  problem  solving 
experimentation  to  investigate  collaboration  methods  and  tools  to  better  improve 
collaboration  on  concept  of  operations  tasks.  That  work  is  documented  in  section  3  of 
this  report. 

Mining  of  primitives  was  conducted  through  decomposition  and  modeling.  The  first 
decomposition  was  performed  on  a  structured  scenario  -  the  Non-Combatant 
Evacuation  Operation,  as  modified  for  an  intelligence  gathering  task.  This  was  then 
modeled  in  an  ontological  tool  (Protege)  to  create  a  taxonomy  of  terms.  This  research 
revealed  that  the  tool  selected  was  not  optimal  for  reporting  results,  and  a  change  of 
tools  was  made,  and  the  NEO  taxonomy  was  modeled  once  again. 

The  second  primitive  model  leveraged  what  was  learned  from  the  NEO  scenario,  but  was 
broader  in  scope.  Instead  of  modeling  a  specific  scenario,  an  organization  was  modeled 
based  on  the  scenarios  performed  by  the  organization.  This  model  was  tested  against 
five  randomly  selected  scenarios  that  organization  would  perform. 

Results  of  the  research  documented  in  this  report,  and  recommendations  for  future 
work  is  provided. 
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2  Introduction  to  Concept  Engineering 


Within  the  DoD,  a  Concept  of  Operations  (CONOPS)  is  a  broadly  known  document. 
Phase  l  of  this  research  discovered  that  there  are  several  standards  for  the  development 
of  a  CONOPS,  however  when  one  looks  at  published  CONOPS,  there  is  little  correlation 
between  the  standards  and  the  final  artifact.  Further,  in  Phase  l,  it  was  discovered  that 
the  term  CONOPS  had  no  meaning  in  the  private  sector.  Yet,  when  the  notion  of  a 
CONOPS  was  described  to  the  private  sector,  they  understood  what  was  meant. 

Based  on  this,  the  researchers  have  begun  to  refer  to  the  practice  of  developing  and 
describing  how  a  new  or  modified  system,  or  collection  of  systems,  would  be  used  when 
deployed,  concept  engineering. 

2.1  Definitions 

CONOPS  -  Concept  of  Operations.  A  detailed  description  of  how  a  new  system  will  be 
deployed  when  in  operational  use 

Model  -  an  abstraction  of  reality,  meant  to  increase  understanding  from  a  specific 
perspective 

Ontology  -  a  formal  representation  of  the  set  of  concepts,  definitions,  and  relationships 
within  a  domain  of  knowledge 

Primitive  -  a  function  or  object  which  is  core  to  a  graphical  programming  language  to 
improve  performance  and  consistency  of  execution  (i.e.,  the  basic  building  blocks) 

Taxonomy  -  in  the  context  of  this  research,  the  practice  of  organizing  and  classifying  of 
information  and  data  and  their  relationships 
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3  Cognitive  Task  Analysis  and  the  NEO  Scenario 


3.1  Literature  Review 

Cognitive  Task  Analysis 

When  team  members  work  together  on  complex  tasks,  they  are  purposeful  in 
coordinating  their  efforts  toward  task  completion.  In  particular,  the  way  team  members’ 
cognitive  processes  unfold  during  the  coordination  of  effort  may  contribute  to  the 
success  of  the  team  (Salas  and  Fiore,  2004).  As  such,  it  is  important  for  researchers  and 
practitioners  to  understand  what  is  going  on  in  the  minds  of  team  members  and  how  to 
facilitate  it.  For  example,  understanding  how  aircraft  crew  members  integrate 
information  in  the  cockpit  may  explain  why  a  crew  fails  to  make  a  corrective  maneuver 
when  an  emergency  situation  arises.  Cognitive  activities,  such  as  information 
integration,  are  team-level  phenomena  that  have  been  termed  team  cognition  (Cooke, 
Gorman,  and  Winner,  2007).  The  measurement  and  analysis  of  team  cognition  may  be 
used  to  improve  team  interactions.  For  example,  by  understanding  what  was  going  on 
cognitively  that  created  the  errors  by  the  aircraft  crew  in  the  emergency  situation,  more 
directed  training  efforts  that  facilitate  crew  communication  in  similar  situations  may  be 
designed  so  that  all  necessary  actions  are  taken  in  the  future.  Herein,  we  discuss 
cognitive  task  analysis  and  the  way  it  has  been  applied  in  the  team-domain  to  examine 
the  underlying  team  cognition  of  taskwork  and  contribute  to  better  team  functioning. 

The  understanding  of  team  cognition  has  been  advanced  by  various  theoretical 
frameworks  that  explain  cognitive  processes  at  the  group-  and  team-level  such  as 
information  processing  (Hinsz,  Tindale,  and  Volrath,  1997),  mental  model  convergence 
(McComb,  2007),  and  transactive  memory  (Yoo  and  Kanawattanachai,  2001).  Based  on 
the  theoretical  underpinnings  of  cognitive  processes  such  as  those  aforementioned, 
specific  types  of  methodologies  have  been  developed  to  capture  and  understand  what  is 
going  on  in  the  minds  of  members  as  they  interact  together.  Cognitive  task  analysis 
(CTA)  is  a  methodology  purposeful  in  discovering  the  way  cognitive  processes  unfold 
during  taskwork  to  produce  behaviors  and  actions  (Crandall,  Kelin,  and  Hoffman, 

2006).  Using  CTA  to  capture  an  individual’s  cognition,  such  as  the  formulation  of  the 
task  in  the  worker’s  mind,  the  way  a  worker  makes  sense  of  information,  and  how  s/he 
is  making  decisions  about  the  task,  may  explain  what  the  worker  does  during  taskwork 
(Chipman,  Schraagen,  and  Shalin,  2000).  CTA  has  also  been  applied  at  the  team-level  to 
study  cognition.  Past  studies  have  applied  the  individual-level  CTA  methods  to  teams 
(e.g.,  Morgan,  Glickman,  Woodard,  Blaiwes,  and  Salas,  1986;  Prince  and  Salas,  1993; 
Stout,  Salas,  and  Carson,  1992).  Research  suggests,  however,  that  team  cognition  may 
be  considered  different  than  just  the  aggregate  of  the  individual  team  members’ 
cognition  (Cooke  et  al.,  2007).  As  such,  applying  methods  from  the  individual-level  may 
overlook  team  member  behaviors  and  actions  due  to  the  aspects  of  interdependence, 
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cooperation,  and  coordination  (Bowers,  Baker,  and  Salas,  1994).  To  attend  to  these 
issues,  recent  research  has  focused  on  eliciting  the  cognitive  process  of  all  team 
members  to  facilitate  the  understanding  of  how  the  team  members’  cognition  effectively 
leads  to  cooperative  efforts  that  produce  outputs  (e.g.,  Klein,  2000). 

The  way  in  which  researchers  apply  CTA  to  capture  team  cognition  has  been  the  subject 
of  numerous  reviews  (e.g.,  Bisantz  and  Roth,  2007;  Blickensderfer,  Cannon-Bowers, 
Salas,  and  Baker,  2000;  2008;  Wei  and  Salvendy,  2004)  and  a  dedicated  website  with 
resources  maintained  by  Aptima  (2010).  The  review  of  CTA  methods  provided  by 
Blickensderfer  and  colleagues  (2000)  and  elaborated  upon  in  other  subsequent  reviews 
suggest  that  team  knowledge  should  be  identified  before,  during  and  after  taskwork.  The 
analysis  of  knowledge  outputs  across  time  may  demonstrate  how  cognition  dynamically 
evolves  to  direct  the  behaviors  and  actions  of  team  members  toward  the  task.  Further, 
the  methods  take  three  different  approaches  to  data  collection.  Table  1  summarizes  the 
different  data  collection  techniques  as  they  vary  across  the  researcher’s  involvement. 

The  issues  regarding  the  different  levels  of  involvement  by  the  researcher  are  discussed 
in  Wei  and  Salvendy  (2004).  While,  all  approaches  have  advantages  and  disadvantages, 
research  suggests  that  multiple  techniques  should  be  applied  in  order  to  obtain  multiple 
outputs.  By  analyzing  multiple  types  of  CTA  data,  the  research  may  be  able  to  better 
triangulate  the  underlying  cognition  of  the  team  (Klein,  2000). 

Table  1:  CTA  Data  Collection  with  Different  Levels  of  Researcher  Involvement 


Data  Collection 
Responsibility: 

Team/team  member(s)  alone 

Researcher-Team/Team 

member 

Researcher  alone 

Approaches  Used 

• 

Questionnaire 

•  Interviews 

•  Ethnography 

• 

Skills  analysis 

(structured  to  semi- 

•  Observation 

• 

Concept  Map 

structured) 

•  Process  diagram 

• 

• 

• 

• 

• 

• 

Problem-solving/Critical 
decision  method 

Running  Commentary 
Rating  data 

Ranking  data 

Sorted  concepts 

Protocol  analysis 

•  Simulation 
interview 

•  Process  tracing 

•  Process  tracing 

•  Textual  coding 

The  use  of  CTA  to  research  team  cognition  in  different  contexts  has  contributed  to  both 
research  and  practice.  Research  has  used  CTA  methods  to  obtain  data  about  what  is 
going  on  in  the  mind  of  team  members  to  establish  team  cognition.  This  knowledge  has 
lead  to  the  improvement  of  training  programs  (e.g.,  Salas  and  Cannon-Bowers,  2001; 
Salas,  Fowlkes,  Stout,  Milanovich  and  Prince,  1999),  interface  design  (e.g.  Naikar, 
2006),  and  software  design  (Shrayne,  Westerman,  Crawshaw,  Hockey,  and  Sauer,  1998) 
for  practitioners.  Thus,  CTA  extends  research  by  providing  a  toolbox  of  methods  for 
understanding  teams  at  a  cognitive  level  and  benefits  the  way  practitioners  go  about 
improving  team  collaboration. 
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3.2  Procedures 

3.2.1  Cognitive  Task  Analysis 

Conducting  a  Cognitive  Task  Analysis  (CTA)  is  anticipated  for  every  scenario  identified 
for  inclusion  in  the  Graphical  CONOPS  Agile  System.  As  previously  mentioned,  the 
purpose  of  conducting  a  CTA  is  to  identity  the  cognitive  tasks  required  of  the  team  to 
complete  the  task.  More  specifically,  the  results  will  help  us  understand  the  cognitive 
processes  the  team  uses  as  they  work  on  the  task,  ascertain  the  mental  models  necessary 
to  be  successful,  inform  the  creation  of  primitives,  and  facilitate  the  development  of  the 
concept  engineering  system  that  will  reduce  the  cognitive  load  associated  with  the  task. 

Data  about  team  cognition  will  be  collected  for  two  different  purposes:  (l)  as  new 
scenarios  are  identified,  we  will  conduct  CTA  with  a  small  number  of  teams  (one  to 
three  depending  upon  the  difficulty  of  the  task  for  the  population  available  for  study)  to 
aid  in  the  creation  of  primitives  and  concept  engineering  system  design  and  (2)  as  the 
concept  engineering  system  is  being  developed,  we  will  conduct  CTA  as  we  run 
experiments  to  test  various  designs  and  design  features  to  identify  the  mental  models  in 
play  and  inform  the  concept  engineering  system  development  process. 

New  Scenario  CTA  Approach 

During  this  phase  of  the  Graphical  CONOPS  project,  we  conducted  case  studies  using 
CTA  to  ascertain  the  approach  that  would  be  most  effective  to  capture  team  cognition 
and  inform  the  creation  of  primitives.  Based  on  these  results  (reported  below),  three 
data  collection  techniques  will  be  used:  concept  maps,  debriefing  interviews,  and 
observation. 

Concept  maps  are  graphical  representations  of  individuals’  information  environments 
with  nodes  representing  the  various  pieces  of  information  available  and  arcs 
representing  the  linkages  between  the  nodes  (Fiol  &  Huff,  1992).  They  have  been  used 
extensively  to  capture  team  cognition  (e.g.,  Marks  et  al.,  2000;  Marks  et  al.,  2002). 

Concept  maps  can  be  completed  by  giving  a  set  of  possible  nodes  to  the  respondents  or 
by  allowing  the  respondents  to  develop  their  own  nodes.  If  nodes  are  given,  the 
relationships  among  the  nodes  are  of  primary  interest  to  the  researchers.  We,  however, 
are  more  interested  in  the  content  of  team  cognition  as  we  investigate  new  scenarios. 
Therefore,  respondents  will  be  allowed  to  devise  their  own  nodes,  thereby  revealing  the 
cognitive  elements  the  team  used  to  complete  the  task. 

The  session  with  the  respondents  begin  with  some  brief  instructions.  The  researchers 
observe  the  teams  as  they  complete  the  task.  Upon  completion  of  the  task,  the 
respondents  receive  instructions  on  how  to  create  concept  maps  and  be  observed  by  the 
researchers  as  they  create  a  concept  map  for  the  task  just  completed.  At  the  end  of  the 
session,  a  debriefing  interview  is  conducted  to  (1)  tap  any  additional  cognitive  elements 
not  uncovered  by  the  concept  map  and  (2)  identify  any  suggestions  for  making  it  easier 
to  complete  this  type  of  task. 
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3. 2. l.i  Concept  Engineering  Experiments  CTA  Approach 

As  the  concept  engineering  system  is  developed,  experiments  will  be  conducted  to 
validate  various  components  of  the  concept  engineering  system.  These  efforts  also 
provide  the  opportunity  to  further  evaluate  the  cognitive  processes  of  the  team  and 
examine  how  the  concept  engineering  system  reduces  the  cognitive  load  experienced  by 
the  team  members  as  they  complete  the  task.  Here,  we  focus  on  the  approach  we  will  use 
to  further  evaluate  the  cognitive  processes  of  the  team.  Based  on  our  case  studies  using 
CTA,  conducted  as  part  of  this  phase  of  the  project,  and  our  experience  examining 
cognition  in  past  research,  four  data  collection  techniques  will  be  used:  questionnaires, 
concept  maps,  process  tracing,  and  textual  coding. 

Pre-  and  post-questionnaires  are  designed  to  tap  the  change  in  respondents’  knowledge 
about  the  task.  They  are  administered  (l)  after  the  respondents  learn  about  the  task  and 
their  individual  responsibilities/expertise  with  respect  to  the  task  but  before  they 
interact  as  a  team  and  (2)  at  task  completion. 

Concept  maps  are  created  by  the  team.  Unlike  in  the  previous  approach,  however, 
respondents  are  given  the  nodes  because  our  purpose  here  is  to  better  understand  how 
they  are  working  through  the  task  using  the  concept  engineering  system  (vs.  the 
identification  of  cognitive  content  when  new  scenarios  are  investigated).  We  anticipated 
that  the  structure  will  change  based  on  the  nature  of  assistance  provided  by  the  concept 
engineering  system  components  under  investigation. 

A  combination  of  process  tracing  and  textual  coding  will  be  used  to  tap  the  mental 
model  content  at  work  during  collaboration.  This  technique  has  been  used  extensively 
by  the  research  team  in  other  projects  (e.g.,  McComb  et  al.,  in  press).  Transcripts  of 
team  sessions  were  created  and  coded.  The  coded  communication  strings  were  then 
analyzed  to  ascertain  how/when  mental  model  convergence  occurs  using  survival 
analysis,  mental  model  content  is  discussed,  etc.  These  results  will  inform  future 
concept  engineering  system  developments. 


3.3  Case  Studies  and  Results 
3.3.1  NEO  Intelligence  Gathering  Task 

The  Non-Combatant  Evacuation  Operation  (NEO)  Intelligence  Gathering  Task  requires 
teams  comprised  of  three  members  with  unique  expertise  (intelligence,  weapons,  and 
environment)  to  identify  the  intelligence  data  required  to  extract  aid  workers  off  a 
remote  island  that  has  been  overtaken  by  hostile  guerilla  forces.  The  complete  task 
documentation  is  included  as  Appendix  A. 
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Task: 

Question  generation 


The  aforementioned 
procedure  for  new  scenarios 
was  used  for  the  case  studies 
conducted  in  this  phase. 

Upon  completing  the  task, 
team  members  (graduate 
students)  worked  together  to 
create  a  concept  map  of  the 
process  they  used  to  complete 
the  task  (see  example  in 
Figure  l).  Next,  the  team 
members  were  debriefed  to 
ensure  that  no  cognitive  steps 
were  omitted  from  the 
concept  map  and  to  discuss 
how  technology  might  have 
assisted  them  in  completing 
the  task.  The  specific 
recommendations  based  on 
this  scenario  and  the  next  one 
are  combined  and  presented 
in  Section  3.4  of  this  report. 

An  additional  element  of  this 
case  study  was  a  critique  of 
the  scenario.  This  scenario 
was  modified  to  represent 
one  step  prior  to  the  original 
NEO  Mission  Planning  Task 
that  will  be  described  next. 

More  specifically,  we  created  a  task  to  identify  the  intelligence  data  necessary  to  devise  a 
mission  plan.  In  the  original  task,  the  intelligence  data  was  provided  to  the  team 
members  and  they  were  required  to  create  a  mission  plan.  The  purpose  of  the 
modification  was  to  make  the  task  more  representative  of  the  types  of  work  undertaken 
by  the  sponsor  of  this  study.  From  that  perspective,  the  modification  was  successful.  The 
revised  task  requires  interdisciplinary  knowledge,  embeds  subjects  in  a  real-world 
scenario,  and  is  relevant  to  the  sponsor’s  domain.  From  a  research  perspective,  the  task 
has  limitations.  In  particular,  the  task  requires  primarily  brainstorming  that  is  difficult 
for  non-experts  to  complete  (and  access  to  experts  for  experimentation  is  virtually 
impossible).  Our  subjects  found  the  uniformity  of  activity  (i.e.,  only  brainstorming)  to  be 
tedious  and  frustrating.  It  was  tedious  because  they  were  brainstorming  lists  for  every 
aspect  of  the  task  and  frustrating  because  they  did  not  have  the  background  to  be  certain 
their  ideas  were  valid.  With  non-motivated  subjects  (e.g.,  undergraduates  earning  extra 
credit  for  a  class,  which  is  the  most  often-used  subject  pool),  they  may  not  fully  engage 


Figure  1:  NEO  Intelligence  Gathering  Task  Concept  Map 
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in  the  activity  to  the  degree  of  the 
graduate  students  who  were  used 
as  subjects  for  the  case  studies  and 
therefore  the  results  would  be 
suspect.  A  task  that  balances  being 
representative  with  being  engaging 
is  recommended  for  future  phases. 

3.3.2  NEO  Mission  Planning 
Task 

The  Non-Combatant  Evacuation 
Operation  (NEO)  Mission  Planning 
Task  was  developed  for  the  Office 
of  Naval  Research’s  Collaboration 
and  Knowledge  Interoperability 
program  (Biron,  Burkman,  & 
Warner,  2008).  While  input  from 
military  personnel  was  used  for  the 
scenario  development,  both 
military  and  nonmilitary  persons 
can  execute  the  unclassified 
scenario.  It  requires  teams 
comprised  of  three  members  with 
unique  expertise  (intelligence, 
weapons,  and  environment)  to 
create  a  mission  plan  to  extract  aid 
workers  off  a  remote  island  that 
has  been  overtaken  by  hostile 
guerilla  forces.  This  task  has  been 
used  extensively  in  studies 
examining  team  cognition.  The 
complete  task  documentation  is 
included  as  Appendix  B. 


Figure  2:  NEO  Mission  Planning  Task  Concept 
Map 


As  with  the  previous  case  study,  team  members  were  asked  to  complete  a  concept  map 
after  they  completed  the  task  (see  Figure  2).  They  were  then  debriefed  to  ensure  that  no 
cognitive  steps  were  omitted  from  the  concept  map  and  to  discuss  how  technology  might 
have  assisted  them  in  completing  the  task.  The  specific  recommendations  based  on  this 
scenario  and  the  next  one  are  combined  and  presented  in  Section  3.4  of  this  report. 
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3.4  Recommendations  from  the  Study 

3.4.1  Tool  Creation 

3. 4. 1.1  System  Structural  Elements 

Two  suggestions  for  structural  elements  are  offered:  a  “parking  lot”  and  breakout  rooms. 
During  the  brainstorming  sessions,  the  team  members  would  have  good  ideas  that  were 
relevant,  but  off  the  current  topic.  A  visible  location  (i.e.,  a  “parking  lot”)  to  store  such 
ideas  would  be  useful.  Good  ideas  could  be  placed  here  while  the  discussion  about  the 
current  topic  is  discussed.  In  this  manner,  neither  the  good  idea  nor  the  flow  of  the 
current  conversation  will  be  lost. 

The  team  members  also  suggested  during  the  debriefing  that  they  would  have  benefited 
from  side  conversations  periodically  that  may  have  been  necessary,  but  disruptive  to  the 
flow  of  the  team’s  conversation.  They  suggested  creating  breakout  rooms  for  side 
conversations.  When  interacting  in  these  rooms,  only  those  participating  in  the 
conversation  would  have  access  to  the  activities  in  the  room.  From  these  rooms,  the 
team  members  could  still  monitor  the  main  team  workspace  and  move  quickly  between 
the  two  spaces  as  the  need  arose. 

3.4. 1.2  Automated  Assistance 

Through  the  debriefing  discussions  following  the  creation  of  the  concept  maps,  the  team 
members  and  research  team  agreed  that  automated  assistance  in  the  form  of 
autonomous  agents  and  an  entry  portal  that  primes  the  system  with  baseline 
information.  Autonomous  agents  could  be  embedded  into  the  concept  engineering 
system  to  augment  team  functioning  and  ease  cognitive  burden  on  the  team  members. 
The  possible  applications  are  numerous.  We  offer  two  suggestions  as  examples.  First, 
agents  could  automatically  update  information  as  databases  change  (e.g.,  weather 
forecasts).  Second,  agents  could  be  used  to  make  connections  among  primitives.  If  the 
weather  forecast  is  updated,  an  agent  for  the  weather  primitive  could  connect  with  an 
agent  for  the  transport  primitive  because  certain  transports  cannot  function  under  some 
weather  conditions. 

A  user-friendly,  entry  portal  that  primes  the  concept  engineering  system  with  baseline 
information  available  from  databases  would  save  a  team  significant  start-up  time  as  they 
initiate  their  collaboration.  For  instance,  once  the  problem  space  and  the  team  are 
identified,  relevant  information  could  be  entered  via  the  entry  portal  and  the  team’s 
workspace  within  the  system  could  be  tailored  to  the  current  project,  including  team 
membership  with  photos,  bios,  notes,  etc.  and  location  geography,  political  climate, 
weather  patterns,  weather  forecast,  etc. 

3.4.2  Primitive  Creation 

During  the  debriefing  interview,  the  team  members  and  researchers  brainstormed  a  list 
of  primitives  that  would  be  useful,  discussed  various  cataloging  schemes  that  might  be 
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practical,  and  suggested  the  need  for  both  summary  and  detailed  information  to  be 
available.  Two  of  the  individuals  participating  in  the  debriefing  interview  were  also  key 
individuals  in  the  primitive  creation  activities.  Therefore,  these  suggestions  have  been 
carried  over  to  that  part  of  the  project  and  incorporated  as  appropriate. 

3. 4. 2.1  Primitive  Dimensionality 

The  group  (i.e.,  team  members  and  researchers)  also  discussed  various  approaches  for 
reducing  the  cognitive  load  on  team  members  through  primitives.  Specifically,  we 
discussed  the  dimensions  of  primitives  and  a  classification  system.  At  a  minimum, 
primitives  should  have  meaning  that  is: 

•  Visual 

•  Intuitive 

•  Simple 

•  Robust 

3 .4 . 2 . 2  Primitive  Classification 

A  classification  system  is  also  necessary.  Through  our  discussion  we  determined  that  a 
subjective  assessment  was  necessary  for  the  primitives.  This  assessment  would  serve  to 
alert  team  members  to  the  usefulness  of  the  information  included  in  the  primitive.  At  a 
minimum,  the  assessment  should  reflect  the  following  meta  data: 

•  Accuracy/Credibility 

•  Relevance/Importance 

•  Effect/Impact 

An  example  of  the  type  of  classification  system  we  envision  is  the  Decision  Making 
Constructs  in  the  Distribute  Environment  (DCODE)  program  (Cowen  &  Fleming,  2005; 
Cowen  &  Fleming,  2006).  DCODE  provides  a  means  of  capturing  implicit  subjective 
assessment  explicitly.  A  combination  of  multiple  bars,  colors,  and  categories  in  a  simple, 
intuitive  design  allow  the  users  to  quickly  classify  the  information  they  provide/review. 
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4  Primitives  and  Ontology  Research 


4.1  Approach 

One  goal  of  this  research  was  to  develop  a  core  set  of  primitives,  or  low  level  nouns  and 
verbs  that  can  be  used  to  construct  a  collection  of  scenarios  to  demonstrate  how  a 
system  would  be  used  when  deployed  in  the  field.  The  initial  problem  investigated  was 
the  Noncombatant  Evacuation  Operation:  Intelligence  Gathering  Scenario  (NEO). 

Once  the  research  was  initiated,  the  sponsor  suggested  the  team  focus  on  a  news 
gathering  agency  instead.  The  team  then  constructed  a  taxonomy  for  that  agency,  based 
on  five  scenarios: 

1.  A  news  agency  deploying  a  reporter  and  support  assets  (truck,  video  camera, 
sound  equipment)  to  a  new  story  (with  time  constraint  option) 

2.  A  news  agency  assigning  a  reporter  to  independently  corroborate  a  news  source 

3.  A  reporter  works  to  recruit  a  new  contact  for  a  story 

4.  A  news  agency  deploying  a  new  reporter  and  support  assets  (truck,  video  camera, 
sound  equipment)  to  follow-up  on  an  existing  (with  time  constraint  option) 

5.  A  news  agency  to  synthesize  multiple  stories  to  create  a  new  edition  for  their 
customers  (with  time  constraint  option) 


4.2  Modeling  the  Ontology 
4.2.1  Tool  Use 

An  ontology  describes  the  concepts  and  relationships  that  are  important  in  a  particular 
domain,  providing  a  vocabulary  for  that  domain  as  well  as  a  computerized  specification 
of  the  meaning  of  terms  used  in  the  vocabulary.  The  initial  plan  was  to  use  Protege, 
which  is  an  open  source  software  (OSS)  product,  written  in  Java,  that  is  freely  available. 
Its  intended  use  is  the  collection  and  organization  of  knowledge  models.  It  provides  a 
collection  of  tools  to  create  and  manage  knowledge  structures.  The  NEO  scenario  was 
decomposed  into  Protege,  but  it  was  found  that  Protege  provided  limited  reporting 
functionality. 

While  this  may  not  be  viewed  as  a  deficiency  of  the  tool  creator  since  their  belief  is  that 
the  model  is  where  the  researcher  should  work,  it  did  not  allow  the  researchers  to 
represent  the  findings  in  this  report  format.  The  researchers  instead  chose  to  represent 
both  scenarios  in  SysML,  a  popular  systems  engineering  model  language,  using  the  tool 
Sparx  Enterprise  Architect  (SparxEA).  Each  scenario  is  investigated  in  detail 
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later  in  this  report;  however  three  high  level  diagrams  of  the  NEO  Scenario,  one 
constructed  using  Protege,  and  the  others  using  SparxEA,  are  represented  in  Figure  3  - 
Figure  5  to  demonstrate  the  differences  in  diagram  creation  and  manipulation. 
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Figure  4:  NEO  Scenario  SysML  Diagram 


Figure  3:  NEO  Scenario  Protege  Diagram 
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The  Protege  model  was  built  to  represent  the  nouns,  properties  and  verbs  associated  with  the 
successful  planning  and  execution  of  the  NEO  Scenario.  Each  noun  was  created  as  a  Protege 
entity  and  added  to  the  model  in  a  hierarchy  to  represent  the  relationships  between  entities. 
Additionally,  properties  and  verbs  can  be  added  to  the  tool  and  associated  with  nouns  in  a 
matrix  style  interface.  Protege  is  a  tool  that  is  used  to  build,  maintain  and  share  ontological 
data.  There  is  a  plug-in  for  Protege  that  takes  all  of  the  entities  and  displays  them  in  a 
hierarchy  tree.  However,  this  tree  hierarchy  will  only  display  entities,  and  will  not  display 
associated  properties  and  verbs.  There  is  currently  no  method  in  Protege  to  display  or  output 
relationships  between  entities  and  the  properties  and  verbs  that  are  associated  to  them. 
Therefore,  the  only  way  to  display  properties  and  verbs  is  to  create  them  as  entities  and  place 
them  in  a  hierarchy  tree.  In  terms  of  the  NEO  Scenario,  the  result  is  a  tree  that  has  NEO 
Primitives  as  its  top  level,  which  decomposes  into  Nouns,  Adjectives  and  Verbs.  Figure  3  only 
displays  the  nouns  of  the  NEO  Scenario;  the  properties  and  verbs  alone  number  well  over  200 
elements,  making  the  full  model  far  too  large  to  display  in  this  report.  Additionally,  while 
Protege  allows  modelers  to  output  these  hierarchy  trees  to  the  depth  of  their  choice,  it  does  not 
allow  for  any  modification  to  the  diagrams. 

The  SparxEA  model  was  built  using  an  object-oriented  approach  in  SysML.  Each  object  (noun 
from  Protege)  was  created  as  a  block  in  SparxEA,  and  these  blocks  were  placed  in  a  Block 
Definition  Diagram,  connected  to  each  other  using  an  aggregation  relationship  (higher  level 
objects  are  aggregates  made  up  of  lower  level  objects.)  One  of  the  many  benefits  of  using 
SparxEA  is  the  variety  of  diagrams  the  tool  can  create  and  the  modifiability  of  each  diagram  to 
meet  the  modeler’s  needs.  For  example,  Figure  4  represents  all  of  the  objects  in  the  top  levels  of 
the  NEO  Primitive  hierarchy.  Each  object  block  was  modified  in  terms  of  its  shape  and  its 
contents  to  create  a  clean,  concise  representation  of  the  scenario.  Figure  5  represents  the 
SysML  model  as  a  Package  Diagram,  in  which  the  top  level  objects  of  the  NEO  scenario  are 
represented  as  packages,  and  their  next  level  objects  are  simply  inserted  as  objects  within  the 
packages.  SparxEA  provides  the  modeler  flexibility  in  displaying  diagrams,  and  the  value  of 
this  is  evident  from  comparing  Figure  3  to  Figure  4  and  Figure  5. 
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bdd  [SysML  Block  Definition]  High  Level  [Object  Packages] 


Operations 

+  Objectives 

+  Operational  Constraints 
+  Operational  Policies 
+  PrimaryObjecti\es 
~~j  +  Rules  of  Eng  ag  ement 
— 1  +  Secondary  Objectives 


Environment 


□  +  Geographic 
+  Physical 
+  Political 
+  Socioeconomical 
+  Support 


Infrastructure 


+  Buildings 

+  Communication  Infrastructure 
+  Electrical  Infrastructure 
+  Land  Use 

+  Medical  Infrastructure 
+  \Afeste  Mang  ement  Infrastructure 


Equipment 


+  CommunicationsEquipment 
+  Medical  Equipment 
~  +PersonalEquipment 
~~  +  Weaponry 


Interfaces 

-  Intelligence 
^  +  Military 

-  Personnel 
I  +  Political 


Risk 


H  +  Contingency  PI  an 
~1  +Risk 

+  Risk  Mitigation  Strategy 


Transportation 


+  Air  Transport 

+  Air  Vehicle 

+  Armored  Fighting  Vehicle 

+  Automobile 

+  Bike 

— 

+  Boat 

+  Fixed  Wing  Aircraft 

+  Land  Transport 

+  Land  Vehicle 

+  Public  Transport 

+  RotaryWing  Aircaft 

+  Sea  Transport 

+  Sea  Vehicle 

+  Submarine 

+  Transport  Animal 

+  Transportation  Infrastructure 

+ Vehicle  Constraints 

+Vehicles 

Figure  5:  NEO  Scenario  SysML  Package  Diagram 
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Additionally,  SparxEA  allows  for  representation  of  the  relationship  between  objects  (nouns) 
and  their  associated  attributes  (adjectives)  and  operations  (verbs)  in  a  concise  manner.  Below 
we  can  see  two  sets  of  diagrams  that  fully  express  a  vehicle  in  the  NEO  Scenario.  Figure  6  was 
built  in  Protege  and  consists  of  four  different  hierarchy  trees,  one  containing  the  vehicle  nouns, 
one  containing  the  vehicle  properties,  one  containing  movement  verbs,  and  one  containing 
movement  properties.  The  use  of  these  four  trees  is  necessary  to  fully  display  the  properties 
and  verbs  of  the  vehicles  in  this  report;  however  they  do  not  represent  any  relationship 
between  the  elements. 

In  contrast,  Figure  7  was  built  using  SparxEA  and  uses  the  object  note  field  to  represent  both 
the  attributes  and  operations  associated  with  the  object,  as  well  as  providing  the  reader  with 
the  definition  of  the  object.  In  this  diagram,  SparxEA  is  able  to  concisely  represent  the 
relationship  between  the  vehicle  objects  and  their  attributes  and  verbs  using  only  five  blocks, 
instead  of  the  30+  elements  required  by  Protege.  Another  feature  of  SparxEA  is  its  use  of 
inheritance.  The  Vehicle  block  in  Figure  7  contains  many  attributes  that  can  be  associated  with 
any  type  of  vehicle.  Because  Land,  Air  and  Sea  Vehicles  are  defined  as  parts  of  the  Vehicle 
aggregate,  they  automatically  inherit  the  Vehicle’s  attributes  and  operations.  This  allows  for  a 
full  expression  of  each  Vehicle  Type’s  attributes  and  operations  without  excessive  repetition. 

A  final  benefit  to  be  noted  in  using  SparxEA  is  its  data  exchange  capability.  Whereas  Protege 
does  not  support  exporting  ontology  data,  SparxEA  allows  the  modeler  to  export  their  model’s 
metadata  in  a  customizable  XML  format.  This  feature  will  prove  to  be  useful  by  allowing  the 
structure  and  properties  of  an  ontology  to  be  transferred  to  other  software  tools. 
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bdd  [SysML  Block  Definition]  Objects  [Vehicle  Example]  J 


«transport» 

Vehicles 


notes 

«Definition» 

Vehicles  are  objects  that  are  used  to  transport  personnel 
and/or  equipment.  In  this  context  they  include  both 
mechanized  and  non  mechanized  transportation  objects. 

«Attributes» 

-hasT  ype 

-hasConstraints 

-hasWeaponry 

-hasLocation 

-hasEquipment 

-hasOperators 

-hasUsage 

-hasFuelType 

-hasFuelCapacity 

-hasFuelQuantity 

-hasRefuel  Method 

-hasT ravel  Speed 

-hasT ravel  Route 

-hasT  rave  I  Distance 

-hasTravelTime 

-h  asT  rave  I  Di  recti  o  n 

«Operations» 

-move  Direction 

-refuelVehicle 

-transportPersonnel 

-transportEquipment 

-dispatchVehicle 

-towVehicle 


-towEquipment 


«vehicle» 

Sea  Vehicle 


«vehicle» 

Land  Vehicle 


«vehicle» 

Air  Vehicle 


Vehicle  Constraints 


notes 

«Definition» 
A  sea  vehicle  is 
any 

transportation 
object  that  uses 
the  water  as  its 
primary  means 
of  transportation 

«Attributes» 

«Operations» 

-sailSeaVehicle 


notes 

«Definition» 
A  land  vehicle 
is  any 

transportation 
object  that  uses 
the  land  as  its 
primary  means 
of  transportation 

«Attributes» 

«Operations» 

-driveLandVehi 

cle 


notes 

«Definition» 
An  air  vehicle  is 
any 

transportation 
object  that  uses 
the  air  as  its 
primary  means 
of  transportation 

«Attributes» 

«Operations» 
-flyAi  rVehicle 


notes 

«Definitions» 

Vehicle  constraints  are 
constrai  nts  prohi  bitti  ng 
or  impairing  the 
operation  of  a  specific 
vehicle. 

«Attributes» 

-haslmpact 

-hasMaxSpeed 

-hasMaxWeight 

-hasMaxDepth 

-hasMaxAltitude 

-hasRange 

-hasCargoCapacity 

-hasPersonnelCapacity 

-hasOperatorCapacity 

-hasSoundProfile 

-hasWeatherConstraints 


«Operations» 


Figure  6:  Vehicle  Primitive  using  Protege 


Figure  7:  Vehicle  Primitive  using  SparxEA 
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4.2.2  Modeling  Methodology 

For  the  purpose  of  developing  the  ontology  using  SparxEA,  an  object  model  was  created  which 
contained  all  of  the  primitives  and  their  associated  definitions,  attributes  and  operations  in  the 
form  of  a  block  definition  diagram.  This  object  model  was  then  split  into  smaller  more 
manageable  sections  while  SparxEA  ensured  all  relationships,  notes,  operations,  and  attributes 
were  mimicked  across  the  diagrams.  The  primitives  were  defined,  labeled  as  objects  (nouns) 
and  the  associated  attributes  (properties)  and  operations  (actions  or  verbs)  were  identified 
where  applicable.  The  relationships  between  the  primitives  were  then  identified  ensuring  that 
the  operations  and  attributes  are  inherited  from  higher  level  primitives  to  the  lowest  level 
primitive.  This  method  was  used  to  represent  the  ontologies  of  the  NEO  and  the  News  Agency 
Scenarios,  as  described  below. 


4.3  NEO  Scenario  /Taxonomy 

4.3.1  Methodology 

The  NEO  Scenario  was  modeled  for  the  purpose  of  identifying  the  relevant  primitives 
associated  with  gathering  intelligence  for  the  successful  planning  and  execution  of  the  NEO 
Scenario.  The  NEO  Scenario,  as  represented  in  Appendix  D,  was  mined  for  its  basic  noun, 
adjective  and  verb  building  blocks,  which  were  added  to  the  ontology.  Relationships  were 
formed  between  these  different  elements,  and  the  ontology  was  further  expanded  to  capture 
missing  elements.  Finally,  definitions  were  applied  to  each  object  and  diagrams  were 
assembled  and  exported  to  this  report. 

Protege  was  initially  used  to  build  the  NEO  Scenario  Ontology.  However,  for  the  reasons 
mentioned  above,  Protege  proved  to  be  an  ineffective  tool  for  this  task,  and  therefore  all 
primitive  modeling  was  transferred  over  to  SparxEA.  In  the  interest  of  space,  this  section  will 
represent  only  the  top  level  SparxEA  diagram.  The  remainder  of  the  diagrams,  and  all  ofthe 
original  Protege  diagrams,  are  available  in  APPENDIX  D.  The  procedure  for  using  SparxEA 
established  above  in  Section  4.2.2  Modeling  was  applied  to  the  News  Agency  Scenario  and  the 
resulting  diagram  can  be  seen  below. 

4.3.2  N  EO  Scenario  Ontology  Diagrams 

Figure  8  represents  the  top  level  decomposition  of  the  NEO  Scenario  Primitives.  It  breaks 
down  the  primitives  into  eight  categories,  Equipment,  Interfaces,  Operations,  Personnel,  Risk 
Assessment,  Infrastructure,  Transportation  and  Environment. 

•  Equipment  represents  the  tools  and  hardware  to  be  used  in  the  scenario.  It  provides 
attributes  specifying  Equipment  Type,  Location  and  Status,  as  well  as  the  necessary 
operations  to  identify  these  attributes. 

•  Interfaces  are  defined  as  instances  where  two  systems  or  persons  interact.  Interface 
attributes  include  the  entities  involved  in  the  interface  and  the  Point  of  Contact  for  these 
entities.  Operations  include  the  actions  taken  at  these  interfaces. 
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•  Operations  is  a  generic  category  to  represent  the  operational  parameters  defining  how 
the  scenario  should  be  carried  out,  and  can  be  further  decomposed  to  Objectives , 
Constraints  and  Policies. 

•  Personnel  refers  to  the  human  objects  involved  in  this  scenario,  and  has  a  wide  variety 
of  attributes  describing  the  people  and  their  associated  attributes  and  operations. 

•  Risk  Assessment  involves  the  identification  and  mitigation  of  risks  associated  with  the 
scenario. 

•  Infrastructure  refers  to  the  physical  and  organizational  structures  that  are  to  be  used 
during  the  scenario,  and  contains  relevant  attributes  such  as  Size,  Age,  Status,  Location, 
Usage  and  Reliability. 

•  Transportation  contains  all  elements  that  facilitate  the  movement  of  persons  or  goods. 
This  category  was  partially  expanded  to  display  vehicle  types  and  constraints  in  Figure 
18,  but  is  further  decomposed  to  include  Transportation  Infrastructure,  which  is 
omitted  from  the  Infrastructure  category  described  above. 

•  Environment  refers  to  the  location  of  the  action  taken  in  the  scenario.  It  can  be 
decomposed  to  include  information  about  the  Physical,  Economic,  Demographic, 
Support,  Political  and  Geographic  Environments. 
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bdd  [SysML  Block  Definition]  High  Level  [1st  Level  links] 

«00» 

NEO  Scenario 
Primitives 


Equipment 

Interfaces 

Operations 

Personnel 

Risk  Assessment 

Infrastructure 

Transportation 

Environment 

notes 

«Definition» 
Equipment 
represents  the  tools 
and  hardware  that 
are  available  for 
use  in  carrying  out 
the  scenario. 

«Attributes» 

-hasType 

-hasLocation 

-hasStatus 

«Operations» 

-identifyEquipment 

-identifyType 

-identifyLocation 

notes 

«Definition» 

An  interface  is 
anywhere  that  two 
systems  or  people 
interact. 

«Attributes» 

-hasPointOfContact 

-hasInterfacedSystems 

-hasInterfacedPeople 

«Operations» 

-interface 

-communicate 

-definelnterface 

notes 

«Definition» 
Operations  are 
the  operational 
parameters 
defining  how 
the  scenario 

should  be 
carried  out 
including 
Objectives, 
Operational 
Policies  and 
Operational 
Constraints. 

«Attributes» 

«Operations» 

notes 

«Defi  nition» 

Personnel  are  objects  that  are  defined  as 
any  human  that  is  involved  in  the  scenario. 

«Attributes» 

-hasCapabilities 

-hasModesOfOperation 

-hasT  raining 

-hasLocation 

-hasWeaknesses 

-hasReadinessLevel 

-hasStatus 

-hasLanguage 

-hasClassifi cation:  asset,  neutral,  - 
opposition,  unknown 
-has  Population 
-hasWeaponry 

«Operations» 

-deployPersonnel 

-gatherPersonnel 

-identifyPersonnelCapabilites 

-identifyPersonnelWeakenesses 

-trainPersonnel 

-contactPersonnel 

-identifyPersonnelStatus 

-identifyPersonnel  Readiness 

-locatePersonnel 

notes 

«Definition» 

Risk  Assessment  refers 
identification  of  risk  and 
its  components,  strategy 
for  mitigating  risk,  and 
emergency  planning  . 

«Attributes» 

-hasRisk 

-hasRiskMitigationStrategy 

-hasContingencyPlan 

«Operations» 

-riskldentifi  cation 

-ri  ski  m  pact  Asse  ssm  e  nt 

-  r  i  skL  i  ke  1  i  h  o  o  d  A  sse  ssm  e  n  t 

notes 

«Definition» 

Infrastructure  is  the 
basic  physical  and 
organizational 
structures  needed  for 
the  execution  of  the 
scenario. 

«Attributes» 

-hasSize 

-hasUsage 

-hasAge 

-hasStatus 

-hasLocation 

-hasReliability 

«Operations» 

-identifylnfrastrcutre 

-explorelnfrastructure 

-utilizelnfrastructure 

-exploitlnfra  structure 

-locatelnfrastructure 

-maplnfrastructure 

notes 
«Definition» 
Transportation  refers 
to  any  element  that 
facilitates  with  the 
movement  of 
personnel  and/or 
equipment. 

«Attributes» 

-hasStatus 

-hasSize 

-hasUsage 

-hasAge 

-hasReliability 

-hasLocation 

«Operations» 
-identifyT  ransportation 
-utilizeT  ransportation 
-movePersonnel 
-moveEquipment 

notes 

«Definition» 
Environment  refers  to 
the  location  of  the 
action  taken  in  the 
scenario  and  the 
location's  attributes. 
Environment  includes 
investigation  of 
physical, 
socioeconomic, 
political,  support  and 
geographic  attributes 

«Attributes» 

«Operations» 

-studyOperationalEnvir 

onment 

Figure  8:  NEO  Scenario  Top  Level  Primitives  Diagram 
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4.4  News  Agency  Scenario  /Taxonomy 
4.4.1  Methodology 

A  News  Agency  Scenario  was  developed  for  the  purpose  of  identifying  the  relevant  primitives 
associated  with  gathering  intelligence  for  the  research,  development,  and  publication  of  news 
stories.  The  gathering  of  intelligence  for  a  news  agency  is  an  activity  that,  although  not  specific 
to  the  sponsor’s  domain  of  interest,  demonstrates  a  number  of  similar  activities  that  are  not 
unfamiliar  to  the  sponsor,  and  as  a  result  many  parallels  and  similarities  between  the  two 
domains  can  be  identified. 

The  development  of  the  News  Agency  Scenario  required  an  in  depth  search  of  online  sources 
covering  a  broad  range  of  search  topics  including:  journalism  techniques  and  approaches,  news 
agency  organization  structures,  news  reporting  techniques  and  approaches,  and  information 
gathering  methodologies,  techniques  and  approaches.  Sources  such  as  www.poynter.org, 
www.pbs.org, www.cvberiournalist.net,  and  www.concernediournalists.org  provided  starting 
points  for  the  generation  of  primitives  for  this  task.  The  information  found  was  consolidated 
and  reviewed  in  detail.  Following  the  review,  the  information  was  mined  for  key  words, 
terminology,  and  phrases  relating  to  the  gathering  of  intelligence  for  a  News  Agency  Scenario. 

Following  the  initial  mining  of  the  information  for  the  appropriate  key  words,  terminology,  and 
phrases,  the  development  of  an  ontology  diagram  for  intelligence  gathering  in  a  News  Agency 
Scenario  could  commence.  An  ontology  diagram  was  created  in  order  to: 

•  Ensure  a  common  understanding  of  the  structure  of  the  intelligence  gathering  process 
within  a  News  Agency  Scenario 

•  Facilitate  the  reuse  of  domain  knowledge 

•  Ensure  all  assumptions  are  explicit 

The  procedure  for  using  SparxEA  established  above  was  applied  to  the  News  Agency  Scenario 
and  the  resulting  diagrams  can  be  seen  below. 
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4.4.2  News  Agency  Scenario  Ontology  Diagrams 


The  News  Agency  Scenario  Object  Model,  which  includes  all  of  the  News  Agency  Scenario 
primitives  with  their  definitions,  attributes  and  operations,  is  too  large  to  be  displayed  here. 
However,  as  with  the  NEO  Scenario  ontology,  the  object  model  diagram  was  simplified  to 
include  only  elements  names,  and  is  displayed  in  Figure  9. 
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Figure  9:  News  Agency  Scenario  Ontology  Simplified  Object  Model 
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As  can  be  seen,  this  object  model  covers  a  set  of  News  Intelligence  Gathering  Primitives 
(Figure  10)  which  can  be  broken  down  into  five  high  level  categories,  News  Medium, 
Information,  News  Organization  Operations,  Transportation  and  Personnel.  These  second 
level  categories  can  be  seen  in  more  detail  in  Figure  10  and  are  further  examined  in  the 
sections  that  follow. 
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Figure  10:  News  Intelligence  Gathering  Primitives  Second  Level  Categories 


Contract  Number:  H98230-08-D-01 71 


Report  No.  SERC-2010-TR-007 
May  31,  2010 


D01,TT02,  RT3 


UNCLASSIFIED 

32 


4’4>2*i  News  Medium 

In  this  scenario,  News  Medium  (Figure  n)  is  the  manner  in  which  news  is  transmitted  to  the 
general  public.  Examples  of  News  Medium  would  include  television  broadcasts,  internet 
websites  and  newspapers. 
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Figure  11:  News  Medium  Primitive  Diagram 

News  Medium  is  further  decomposed  to  Print  Media,  which  includes  Newspapers  and 
Magazines,  and  Broadcast  Media,  which  includes  Television,  Radio  and  Online  news.  Each  of 
these  examples  contains  the  attribute  Type,  specifying  the  scope  of  the  media,  both  in  terms  of 
news  types  (sports,  politics,  entertainment,  etc.)  and  the  community  that  those  types  cater  to 
(local  or  national).  The  higher  level  News  Medium  block  also  contains  a  Source  attribute, 
which  is  inherited  by  all  of  its  children,  and  documents  the  source  of  the  News  Medium.  News 
Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT3 

Report  No.  SERC-2010-TR-007 
May  31,  2010 


UNCLASSIFIED 

33 


Media  represents  where  the  Information  collected  by  the  News  Agency  is  presented  to  a 
customer. 

4 .4 . 2 . 2  Information 

Information  is  by  far  the  largest  category  within  the  News  Intelligence  Gathering  Primitive, 
which  is  appropriate  given  that  information  is  the  focus  of  any  intelligence  collection  task.  In 
addition  to  having  the  most  number  of  decomposed  elements,  Information  also  contains  some 
of  the  most  extensive  listings  of  attributes  and  operations.  From  the  definition  in  Figure  12  it 
can  be  seen  that  Information  is  expanded  to  include  both  the  types  of  information  being 
conveyed,  as  well  as  the  sources  of  this  information  and  the  strategies  used  to  solicit  these 
sources.  Included  in  the  high  level  Information  block  are  attributes  assessing  the  information’s 
validity  ( Accuracy  and  Objectivity )  as  well  as  the  relevant  Target  of  the  information,  and  its 
Flow.  Again,  these  attributes  are  inherited  by  lower  level  blocks,  which  is  discussed  below. 

Information  is  decomposed  to  include  its  Strategy,  Types  and  Sources.  Each  lower  level 
Information  category  has  attributes  and  operations  associated  with  it  specifically,  and  will  be 
expanded  below. 

Information  Strategy  is  defined  as  the  plan  or  method  with  which  information  is  collected, 
analyzed  and  managed  before  it  can  be  reported  to  the  customer.  Because  Information 
Strategy  is  actively  controlled  by  a  News  Agency,  it  has  a  number  of  attributes  and  operations 
associated  with  it,  as  can  be  seen  in  Figure  12.  One  of  the  most  important  attributes  is  the 
Constraint.  These  Constraints  are  crucial  to  an  organization  because  they  establish  the  playing 
field  within  which  the  organization  will  operate.  Without  fully  understanding  Information 
Strategy  Constraints,  a  News  Organization  will  struggle  to  complete  its  objectives.  Additional 
critical  aspects  of  the  News  Agency  Scenario  are  the  Investigation,  Validation  and  Fact 
Checking  operations.  The  effects  of  an  immature  story  verification  strategy  can  severely 
embarrass  an  organization  and  can  make  it  vulnerable  to  attack  by  both  its  competitors  and 
customer  base.  This  was  the  case  with  Jayson  Blair  of  the  New  York  Times,  whose  extensive 
plagiarism  and  fabrication  of  sources  and  story  content  went  completely  undetected  because  of 
a  lack  of  proper  fact  checking  procedures,  and  brought  embarrassment  to  one  of  the  world’s 
premier  news  outlets.  Development  of  Information  Strategy  is  crucial  to  an  organization  as  it 
will  impact  nearly  every  other  aspect  of  News  Intelligence  Gathering. 

Information  Types  (Figure  13)  are  defined  as  the  variety  of  information  that  can  be  used  to 
assemble  news  stories  for  distribution  to  a  customer.  The  types  investigated  in  this  News 
Agency  Scenario  include  New,  Background,  Continuing  and  Existing  Information,  which  can 
be  seen  below. 
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Figure  12:  Information  Primitive  Diagram 
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Figure  13:  Information:  Information  Types  Primitive  Diagram 

New  Information  relates  to  information  that  was  previously  unknown,  whereas  Background 
Information  is  information  that  is  exposed  through  research  of  relevant  subject  matter, 
Continuing  Information  is  information  that  updates  a  news  story  and  Existing  Information  is 
information  that  could  have  been  reported  on  in  the  past  by  the  News  Agency.  Each  type  has  a 
NewsSource  attribute  referencing  where  the  specific  information  came  from.  This  attribute 
leads  directly  to  the  next  Information  subclass  from  Figure  12,  Information  Sources. 

Information  Sources  are  the  people,  organizations,  artifacts  and  systems  from  which  a  News 
Agency  collects  information.  Information  Sources  has  many  key  attributes  including  Location 
of  the  source,  the  source  Type  as  either  primary  or  secondary,  the  Accessibility  of  the  source, 
both  in  terms  of  first  and  continuing  communication,  and  source  Reliability.  Each  of  these  is 
investigated  through  a  complimentary  operation  that  represents  the  action  needed  to  collect 
these  attributes.  As  seen  in  Figure  14,  Information  Sources  can  be  decomposed  into  seven 
major  categories:  Video  Capture,  People  in  the  News,  Independent  News  Creator,  External 
Systems,  Other  News  Agencies,  Information  Archives  and  Contact. 
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Figure  14:  Information:  Information  Sources  Primitive  Diagram 

Elements  of  Information  Sources  that  are  becoming  a  larger  part  of  the  way  News  Agencies 
operate  are  the  Video  Capture  and  External  Systems.  With  increases  in  video  device 
technology,  more  businesses  are  installing  security  cameras,  more  individuals  own  personal 
camcorders,  and  more  law  enforcement  agencies  are  employing  city  wide  surveillance 
networks,  all  of  which  can  be  potentially  rich  sources  of  news  information.  Additionally,  in 
recent  years,  a  growing  segment  of  the  news  delivery  market  has  been  the  so  called  news 
aggregators  which,  in  this  ontology,  are  considered  to  be  External  Systems.  These  entities  are 
able  to  collect  news  reports  from  smaller  local  news  agencies  and  deliver  then  rapidly  via  the 
internet  to  a  national  market  of  news  readers.  Services  like  Google  News  can  be  used  as  News 
Agencies  as  a  “one-stop  shop”  of  headlines  from  around  the  world,  allowing  the  Agency  to 
target  specific  stories  and  dispatch  resources  to  collect  Continuing  Information.  These  two 
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elements  are  a  growing  part  of  the  News  Agency’s  possible  Information  Sources.  The  central 
element  of  the  Information  Sources,  however,  is  represented  by  the  Contact  block,  which  is 
expanded  in  Figure  15. 
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Figure  15:  Information:  Information  Sources:  Contact  Primitive  Diagram 

The  top  level  Contact  block  defines  a  contact  as  a  person  who  is  able  to  provide  Information 
relating  to  a  news  story.  Contacts  have  attributes  that  inform  the  news  agency  to  their 
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Location,  Rank,  Type  and  their  classification  as  a  New  or  Existing  Contact.  These  are 
accompanied  by  attributes  of  Accessibility  and  Reliability  which  are  inherited  from 
Information  Sources  and  are  used  to  assess  the  quality  of  Information  that  a  Contact  provides. 
Contact  is  broken  down  into  four  broad  types  of  contacts,  Subject  Matter  Expert,  Whistle 
Blower,  Tipster  and  Official  Spokesperson.  Each  of  these  can  be  further  broken  down  based  on 
the  Type  attribute. 

4.4.2.3News  Organization  Operations 

Stepping  back  to  the  News  Intelligence  Gathering  Primitives  Second  Level  Categories,  the  next 
decomposed  element  is  titled  News  Organization  Operations,  and  can  be  seen  in  Figure  16. 
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bdd  [SysML  Block  Definition]  News  Organization  Operations  [New 

s  Oraanization  Ooera 

ions_Master] 

«block» 

News  Intelligence 
Gathering  Primitives 

«block» 

News  Organization  Operations 


notes 

«Defi  nition» 

News  Organization  Operations 
refers  to  the  various  operational 
considerations  within  the  News 
Agency  e.g.  operational  policies, 
operational  environment, 
operational  constraints,  and 
operational  objectives. 


«block» 

«block» 

«block» 

«block» 

Operational  Policies 

Operational  Environment 

Operational  Constraints 

Operational  Objectives 

notes 

notes 

notes 

notes 

«Definition» 

«Definition» 

«Defi  nition» 

«Defi  nition» 

Operational  policies  refer  to  those 

Operational  Environment  refers  to 

Operational  constraints  refer  to  the 

Operational  Objectives  refer  to  the 

organizational  written  statements 

the  technology,  resources,  and 

restrictions  the  news  agency 

goals  of  the  news  agency  in  order  to 

that  communicate  the  agency's 

processes  that  the  news  personnel 

personnel  must  adhere  to  when 

successfully  realize  a  news  story. 

intent,  objectives,  requirements, 

must  utilize  in  order  to  realize  a 

meeting  deadlines  e.g.  for  print 

responsibilities,  and  standards,  in 

news  story. 

media  spatial  and  temporal 

«Attributes» 

order  to  provide  guidance  to  the 

constraints  exist,  a  story  may  be  time 

-hasType:  Primary,  Secondary 

personnel  regarding  the  operation 
of  the  news  agency. 

«Attributes» 

critical,  ...etc. 

«Operations» 

«Attributes» 

«Attributes» 

«Operations» 

-accessConstraints: 

-listsObjectives:  Primary,  Secondary 

-studyOperational  Environment 

-hasEditorialParameterConstraints: 

-listsRequirements 

temporal,  spatial 

-listsResponsibilities 

-influencelnformationStrategy: 

-specifiesStandards 

-timeConstraints: 

-stateslntent 

environmentalCOnstraints: 

-ti  meOf  DayConstrai  nts: 

«Operations» 

-weatherConstraints: 

«Operations» 

-identifyConstraints: 

Figure  16:  News  Organization  Operations  Primitive  Diagram 

News  Organization  Operations  is  a  broad  category  that  includes  many  aspects  related  to  how 
the  News  Agency  operates.  This  ontology  breaks  down  News  Organization  Operations  into 
Operation  Policies,  Environment,  Constraints  and  Objectives.  Operation  Policies  are  the 
guidelines  that  News  Agency  Personnel  should  follow  so  that  the  News  Agency  can  successfully 
accomplish  its  objectives.  Operation  Environment  is  further  decomposed  and  will  be  discussed 
below.  Operation  Constraints  lists  the  restrictions  that  News  Agency  Personnel  must  adhere  to 
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when  creating  news  stories,  such  as  Time  deadlines.  Attributes  of  Operational  Constraints 
include  the  different  types  of  constraints,  such  as  Editorial  Parameter  Constraints, 
Environmental  Constraints,  Time  Constraints  and  Time  of  Day  Constraints.  In  order  for  the 
News  Agency  to  be  able  to  understand  these  Constraints,  an  Identify  operation  is  included. 
Finally,  the  Operational  Objectives  refer  to  the  primary  and  secondary  objectives  of  the  News 
Agency,  which  will  usually  include  the  presentation  of  an  accurate  news  report  to  a  customer. 

The  Operational  Environment  object  is  further  decomposed  to  include: 

•  News  Communication  Modes  -  Methods  of  communicating  within  and  outside  of  the 
News  Agency 

•  Resources  -  Include  technology,  personnel  and  equipment  that  the  News  Agency  needs 
to  employ  to  successfully  report  on  a  news  story. 

•  News  Equipment  -  Deals  specifically  with  the  equipment  needed  to  collect  and  report 
news  stories,  such  as  lights,  cameras,  microphones,  etc. 

•  News  Construction  Process  -  A  predetermined  set  of  steps  to  be  followed  in  the 
assembling  of  a  news  story. 

•  Risk  Management  Process  -  The  News  Agency’s  strategy  for  reducing  the  risk 
associated  with  assembling  and  reporting  on  news  stories.  Risk  Management  Process  is 
of  special  interest  as  it  will  affect  the  News  Agency  in  all  aspects  of  its  operation. 
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bdd  [SysML  Block  Definition]  Operational_Environment_Master  [Operational_Environment_l\/ 


«  block» 

News  Communication 
Modes 


notes 

«Defi  nition» 

News  Communication 
Modes  refers  to  the  modes 
of  communication 
adopted  within  and 
external  to  the  News 
Agency.  It  specifically 
relates  to  how  the  agency 
communicates  with 
sources,  with  other 
personnel,  with  other  news 
agencies 

«Attributes» 

-hasType:  Internal 
Communication  Modes 
External  Communication 
Modes 

«Operations» 


«block» 

Operational  Environment 


notes 

«Definition» 

Operational  Environment  refers  to  the  technology, 
resources,  and  processes  that  the  news  personnel  must 
utilize  in  order  to  realize  a  news  story. 

«Attributes» 


notes 

«Definition» 

Resources  refer  to  the 
technology,  personnel, 
equipment,  etc  that  the 
News  Agency  personnel 
need  access  to  in  order  to 
successfully  meet  their 
news  story  goals 

«Attributes» 

-isEquipment:  News 

Equipment, 

Communication 

Equipment 

-isPersonnel:  Internal, 

External 

-isVehicleType:  Ground, 
Air 

-hasStatus:  Available, 
Unavailable 
-hasLocation:  Local, 
Remote 

«Operations» 

-identifyResources 


«block» 

News  Equipment 


notes 

«Definition» 

News  Equipment  refers  to 
the  specific  equipment 
needed  to  produce  a  news 
story  e.g.  Sound 
Equipment,  Networked 
Computers,  ...etc 

«Attributes» 

-hasType:  Networked 
Computers 
Telecommunications 
Equipment,  Camera 
Equipment,  Sound 
Equipment,  Lighting 
Equipment 
-hasStatus: 

-hasLocation: 

«Operations» 
-identifyEquipment: 
-identifyEquipmentT  ype: 
-identifyEquipmentStatus: 
-identifyEquipmentLocatio 


«block» 

News  Construction  Proces;; 


notes 

«Definition» 

News  construction  process 
refers  to  the  series  of 
prescribed  steps 
implemented  in  order  to 
achieve  the  news  story  goal 
The  process  may  not  requiri 
that  all  steps  are  completed. 

«Attributes» 

-h  asl  nf  o  rm  ati  o  n  Type : 
Background  Information, 
Continuing  Information,  Ne|/\ 
Information 

«Operations» 
-editlnformation 
-interpretlnformation 
-targetlnformation 


«block» 

Risk  Management  Process 


notes 

«Definition» 

Risk  Management  Process 
refers  to  the  series  of  steps 
implemented  in  order  to 
reduce  the  risks  associated 
with  producing  a  news 
story  that  has  to  be 
retracted  at  a  later  date 
due  to  falsly  reported 
information. 

«Attributes» 

«Operations» 

-riskldentifi  cation 

-ri  ski  m  pact  Assessm  ent 

-riskLi  kel  i  hood  Assessment 


Figure  17:  News  Organization  Operations:  Operational  Environment  Primitive  Diagram 

Each  sub  class  of  Operational  Environment  (Figure  17)  contains  several  relevant  attributes 
such  as  Type,  Location,  and  Status,  and  associated  operations.  The  Operational  Environment 
represents  an  important  segment  of  the  News  Agency  Intelligence  Gathering  Scenario  in  that  is 
contains  many  of  the  “rules  and  regulations”  and  the  standard  operating  procedures  involved 
in  news  reporting. 
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4.4*2‘4Transportation  and  Personnel 

The  final  two  objects  from  the  News  Intelligence  Gathering  Primitives  Second  Level  Categories 
are  Transportation  and  Personnel.  These  two  elements  will  be  discussed  together  since  they 
strictly  represent  tangible  physical  items  and  are  fairly  straight  forward.  Transportation  is 
represented  in  Figure  18  and  is  decomposed  into  Transportation  Vehicles  and  Infrastructure. 
Infrastructure  refers  to  installations  allowing  Transportation  Vehicles  to  operate  such  as 
roads,  highways,  railroad  tracks,  etc.  The  Transportation  Vehicle  object  is  broken  down  into 
Aircraft  and  Ground  Vehicles,  which  contain  vehicle  attributes  such  as  Max  Speed,  Max 
Elevation,  Vehicle  Equipment,  and  Operator  Personnel,  and  operations  representing  vehicle 
motion,  as  well  as  Dispatch  and  Identify  vehicles. 

The  Personnel  (Figure  19)  object  includes  both  News  Agency  Personnel  and  News  Consumers. 
News  Agency  Personnel  include  Reporter,  Editor,  Researcher,  Writer,  Technical  Personnel 
and  Vehicle  Controller.  Each  has  attributes  describing  their  Status,  Location,  Capabilities, 
Skills  and  Training  Level.  Additionally,  each  contains  operations  that  allow  for  the 
identification  of  these  attributes.  At  the  head  of  each  arrow  linking  specific  personnel  to  the 
Personnel  object  are  a  set  of  numbers  that  indicate  multiplicity.  These  numbers  reflect  the  fact 
that  each  News  Story  must  have  a  Reporter,  Vehicle  Controller,  Editor,  Researcher  and  Writer 
associated  with  it,  however  there  need  not  be  a  Consumer  or  Technical  Personnel. 
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bdd  [SysML  Block  Definition]  Transportationlnfrastructure  [Transportation_lnfrastructure_Master] 


«biock» 

Aircraft  Vehicle 


«block» 

News  Intelligence  Gathering  Primitives 


«block» 

Transportation 


notes 

«Definition» 

Transportation  refers  to  any  element  that 
dealswith  the  movement  of  personnel 
and/or  equipment. 

«Attributes» 

-hasStatus: 


«block» 

Transportation  Vehicles 


notes 

«Definition» 

Transportation  Vehicles  refers  to  an 
elements  that  is  used  to  move  personnel 
and  equipment. 

«Attributes» 

--hasEquipment:  Sound  Equipment, 
Camera  Equipment,  Lighting  Equipment 
-hasLocation:  local,  remote 
-hasPersonnel:  News  Reporter,  Sound 
Crew,  Camera  Crew,  Lighting  Crew 
-hasType: 

«Operations» 

-moveBackward:  mph 
-moveForward:  mph 
-identifyLocation: 

-dispatchVehicle: 

identifyVehicleType: 


«block» 

Ground  Vehicle 


«block» 

Infrastructure 


notes 

«Definition» 

Infrastructure  refers  to  fixed  installations 
that  allow  a  vehicle  to  operate. 

«Attributes» 


«Operations» 


notes 

«Definition» 

This  represents  any  vehicle  within 
the  NewsOrg  that  can  transport 
both  personnel  and  equipment, 
between  the  News  Agency  (or 
other  remote  locations)  and  News 
Source  sites,  while  travelling  in 
the  air. 

«Attributes» 

-hasMaxAltitude: 

-hasMaxSpeed: 

-hasMinSpeed: 

-hasType:  Helicopter 

«Operations» 


notes 

«Definition» 

This  represents  any  vehicle 
within  the  NewsOrg  that  can 
transport  both  personnel  and 
equipment,  between  the 
News  Agency  (or  other  remote 
locations)  and  News  Source 
sites,  while  travelling  on  land. 

«Attributes» 

-hasMaxSpeed: 

-hasMinSpeed: 

-hasType:  Car,  Van,  Truck 

«Operations» 


«block» 

Water  Vehicle 


notes 

«Definition» 

This  represents  any  vehicle 
within  the  News  Org  that  can 
transport  both  personnel  and 
equipment,  between  the 
News  Agency  (or  other  remote 
locations)  and  News  Source 
sites,  while  travelling  in  water. 

«Attributes» 

-hasMaxSpeed: 

-hasMinSpeed: 

-hasType: 

-hasMaxDepth: 

«Operations» 


Figure  18:  Transportation  Primitive  Diagram 
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bdd  [SysML  Block  Definition]  Personnel  [Personnel_Master] 


«block» 

News  Intelligence 
Gathering 
Primitives 


«block» 

Personnel 


notes 

«Definition» 

Personnel  refers  to  both  News  Agency 
Personnel  (the  people  that  work  for  the 
News  Agency),  and  News  Consumers  (the 
people  that  read,  watch,  ...etc  the 
published  or  broadcasted  news). 

«Attributes» 

-hasLocation:  Local,  Remote 

hasType: 

hasStatua 

«Operations» 

-identifyPersonnel 
-identifyPersonnelType: 
-identifyPersonnelStatus: 
-identifyPersonnelLocation: 
-dispatchPesonnel: 


«block» 

«block» 

«block» 

«  block» 

«block» 

«block» 

«block» 

News  Consumer 

News  Reporter 

Technical  Personnel 

Vehicle  Controller 

News  Editor 

News  Researcher 

News  Writer 

notes 

notes 

notes 

notes 

notes 

notes 

«Definition» 

«Definition» 

notes 

«Definition» 

«Definition» 

«Definition» 

«Definition» 

-This  is  the  person  that 

-These  are  the  News  Agency 

«Definition» 

-This  is  the  person  that  edits 

-This  is  the  person  that 

-This  is  the  person  that 

reports  (broadcast)  the 

personnel  with  specific 

the  News  Story  before 

researches  the  news  story 

reports  (print)  the  researched 

person  that  the  constructed 

researched  information. 

technical  skills e.g. 

transports  equipment  and 

publication  (print,  broadcast, 

and  provides  the  background 

information. 

news  (print  or  briadcast 

videographera 

other  News  Agency 

...etc). 

information  to  the  News 

media)  istargetted  at. 

«Attributes» 

personnel  between  the  News 

Writers  or  News  Reportera 

«Attributes» 

-hasCapabilities 

«Attributes» 

Agency  and  News  Source 

«Attributea» 

-hasCapabilities 

«Attributes» 

-hasS  kills 

-hasCapabilities 

sitea 

-hasCapabilities 

«Attributes» 

-hasS  kills 

-hasType:  News  Listener, 

-hasTrainingLevel 

-hasS  kills 

-hasS  kills 

-hasCapabilities 

-h  asT  ra  i  n  i  n  g  Le  ve  1 

News  Reader,  News  Viewer 

-hasTrainingLevel 

«Attributes» 

-hasTrainingLevel 

-hasS  kills 

«Operations» 

-hasType:  Videographer, 

-hasCapabilities 

-hasTrainingLevel 

«Operations» 

-identifyCapabilities: 

Photographer,  Sound 

«Operations» 

-idenitfyCapabilities: 

-accessNews: 

Engineer 

-hasTrainingLevel 

-idenitfyCapabilities: 

«Operations» 

-MstenToNews: 

-re  ad  News: 

«Operations» 

«Operations» 

-idenitfyCapabilities: 

-search  News: 

-identifyCapabilities: 

-  operateVehicle: 

-viewNews: 

-idenitfyCapabilities: 

Figure  19:  Personnel  Primitive  Diagram 
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5  The  Scenario  Generator  Interface 


One  example  of  a  concept  engineering  interface  that  was  introduced  in  Phase  l  was  the 
interface  found  on  StarLogo  NXT.  The  concept  is  to  have  snap  together  “lego”  like  building 
blocks  to  graphically  generate  a  scenario  or  story.  A  post  processor  will  create  an  executable 
model  from  the  graphically  constructed  scenario.  As  part  of  this  effort,  we  have  constructed  a 
similar  web-based  prototype  to  demonstrate  this  concept  which  is  discussed  next. 

5.1  Scenario  Introduction 

To  further  investigate  an  approach  to  scenario  generation,  and  to  test  the  robustness  of  the 
created  news  agency  taxonomy,  the  team  came  up  with  five  representative  scenarios. 

1.  New  Story:  A  news  agency  deploying  a  reporter  and  support  assets  (truck,  video  camera, 
sound  equipment)  to  a  new  story  (with  time  constraint  option) 

2.  Corroborate  Source:  A  news  agency  assigning  a  reporter  to  independently  corroborate  a 
news  source 

3.  New  Contact:  A  reporter  works  to  recruit  a  new  contact  for  a  story 

4.  Follow-up:  A  news  agency  deploying  a  new  reporter  and  support  assets  (truck,  video 
camera,  sound  equipment)  to  follow-up  on  an  existing  (with  time  constraint  option) 

5.  Information  Synthesis:  New  agency  to  synthesize  multiple  stories  to  create  a  new 
edition  for  their  customers  (with  time  constraint  option) 

5.1.1  New  Story 

In  this  scenario,  a  news  agency  has  a  lead  on  a  new  story,  and  deploys  a  reporter  and  the 
supporting  equipment  and  personnel  to  support  the  reporter  while  covering  the  story. 

5.1.2  Corroborate  Source 

Considers  the  scenario  in  which  a  reporter  is  assigned  to  independently  corroborate  a  news 
source. 

5.1.3  New  Contact 

In  the  news  business,  many  a  story  is  developed  via  sources.  In  this  scenario,  a  reporter  works 
to  recruit  a  new  source/contact  for  a  story. 

5.1.4  Follow-up 

Once  a  story  has  been  covered,  it  is  not  unusual  to  do  a  follow-up  story.  This  scenario  depicts  a 
news  agency  deploying  a  different  reporter  and  support  assets  to  follow-up  on  an  existing 
story. 
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5.1 .5  Information  Synthesis 


One  form  of  news  delivery  is  via  the  printed  medium.  This  form  of  report  requires  the  news 
agency  to  synthesize  multiple  stories  to  create  a  new  edition  for  their  customers. 

Appendix  C  contains  a  list  of  each  primitive  used  in  creating  these  scenarios. 

5.2  Scenario  Generator  Interface 

5.2.1  Scenario  Generator 

The  Scenario  Generator  was  designed  as  a  demonstration  of  what  a  possible  user  interface 
would  look  like  in  a  concept  engineering  environment.  It  was  assembled  as  a  Java  applet  to 
make  it  platform  independent  and  accessible  via  the  web. 

The  Scenario  Generator  is  shown  in  Figure  20. 


Choose  Piece  Choose  Color  |  Scenarios  f+  j|  Update  Tab  Name  |  Delete  Current  Tab  | 

SYSTEMS  ENGINEERING  Agile  CONOPS 
Research  Center 

Scenario  1  Scenario  2  Scenario  3  Scenario  4  Scenario  5 


trash 


Figure  20:  Scenario  Generator  Interface 


Like  today's  Internet  browsers  the  tool  was  designed  to  support  multiple  work  spaces  through 
the  use  of  different  tabs.  The  tabs  are  listed  above  the  black  workspace  and  can  be  created, 
deleted  or  renamed  to  allow  the  modeler  a  full  range  of  flexibility. 

Six  basic  building  blocks  were  created,  Run,  Conditional,  Boolean,  Number,  Single  Slot  and  No 
Slot,  as  seen  in  Figure  21. 
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Number  Choose  Color  |  j  io  Generator  Building  Blocks  +  |  Update  Tab  Name  |  Delete  Current  Tab  | 


SYSTEMS  ENGINEERING  Agile  CONOPS 
Research  Center 

Scenario  Generator  Building  Blocks 


boolean 


dummy 


number 


single  slot 


no  slot 


Figure  21:  Scenario  Generator  Building  Blocks 


The  blocks  were  designed  to  be  flexible  in  what  they  represent,  allowing  for  changes  to  be  made 
to  their  name  and  color  in  the  area  above  the  workspace.  Colors  can  be  chosen  from  a  color 
palette,  or  specified  using  RGB  or  HSB  values  using  the  color  pop-up  seen  in  Figure  21.  Blocks 
can  also  be  removed  from  the  work  space  using  the  trash  can,  seen  at  the  bottom  of  the 
workspace  in  Figure  20. 

Like  the  Starlogo  tool,  the  edges  of  some  blocks  are  specific  shapes  that  fit  into  other  blocks, 
allowing  them  to  be  snapped  together  and  moved  as  a  single  unit.  This  way  the  modeler  is  able 
to  create  chains  out  of  related  concepts.  Some  of  the  blocks  in  this  version  of  the  tool  (Dummy 
and  Fallback)  are  not  able  to  be  snapped  together,  and  were  created  to  simply  represent  the 
possibility  of  future  block  shapes. 
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The  modeling  process  for  the  News  Agency  scenarios  was  relatively  easy  in  terms  of  renaming 
and  snapping  together  blocks,  but  slightly  more  complicated  when  it  came  to  color  coding 
blocks.  The  Scenario  Generator  modeling  process  will  be  discussed  below,  using  Scenario  l  as 
an  example 

Scenario  l:  A  news  agency  deploying  a  reporter  and  support  assets  to  a  new  story 

The  first  action  taken  in  the  modeling  process  was  to  create  Scenario  Generator  tabs  for 
Scenarios  1-5,  thus  creating  five  independent  work  spaces  (Figure  22).  Next,  colors  were 
chosen  that  were  to  represent  certain  primitives.  The  Scenario  Generator  only  supported  white 
text  therefore  there  were  a  limited  number  of  colors  that  would  be  readable  on  the  screen.  As  a 
result,  colors  were  chosen  for  groups  of  related  primitives.  All  attributes,  which  were  to  be 
represented  by  the  number  block,  were  set  as  a  light  grey  color.  The  short  name  of  each 
scenario  is  a  run  block  and  colored  black  to  appear  as  if  it  is  part  of  the  background,  and  reduce 
clutter  in  the  work  space.  From  there,  the  primitives  listed  in  Appendix  C  that  represented 
similar  concepts  were  assigned  a  color.  For  example,  all  primitives  dealing  with  Operational 
aspects  were  assigned  the  golden  color  and  all  Personnel  related  primitives  were  colored 
orange.  Some  primitive  color  groupings  were  less  obvious,  such  as  dark  blue  which  represents 
information  gathering  primitives,  or  light  green,  which  represents  information  assessment 
primitives.  Finally,  some  of  the  primitives  did  not  fit  neatly  into  a  color  category,  and  were 
assigned  the  turquoise  color,  such  as  dispatch  and  report. 
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Scenario  1  Scenario  2  Scenario  3  Scenario  4  Scenario  5 


Time  Environmental 


New  Mews  Story  Identity  Operational  Constraints 


identity  Operational  Objectives 


Operational  Policies 


Mine  Information  Archives 


Gather  Background  Information 


Study  Operational  Environment 


Identity  Information  Sources 


Reliability  Accuracy 


Evaluate  Infromation  Source 


Identity  Information  Strategy 


Reqs  Type  Reqs  Type  Reqs  Type 

Identity  Transport  Vehicle  I  Equipment  I  Personnel  H 


Identify  TransportVehicle 

Capabilities 


Status 

Location 

Equipment  1 

Status 

Location 

Personnel 

Identity  Personnel 


Identity  News  Story 


Location  Accessibility 


Type  Operator  Type  Numbers  Type 

Dispatch  TransportVehicle  I  Equipment  I  Personnel 


Gather  Information 


Analyze  Information 


Statistical  Accuracy 


Fact  Check  Information 


Report  Information 


Figure  22:  Scenario  1  -  New  News  Story 
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Once  each  primitive  in  the  scenario  list  was  assigned  a  color,  the  modeling  began.  As 
mentioned  above,  the  run  block  was  used  to  label  the  scenario  using  its  short  name,  and  was 
placed  in  the  far  left  to  allow  the  remainder  of  the  blocks  to  be  snapped  to  it.  The  first 
primitives  in  each  scenario  listing,  operations,  were  assigned  to  a  conditional  block,  labeled 
and  colored  appropriately,  and  snapped  to  the  run  block  in  the  order  in  which  they  appeared  in 
the  scenario  listing.  From  here,  objects,  the  second  primitive  in  each  scenario  listing,  were 
assigned  either  the  boolean,  single  slot  or  no  slot  blocks  based  on  the  number  of  attributes 
associated  with  them.  These  blocks  were  labeled,  colored,  and  snapped  to  their  appropriate 
location,  either  into  a  conditional  block  or  after  another  object  block.  Finally,  where  needed, 
the  number  block  was  labeled  as  the  appropriate  attribute  and  inserted  into  the  object  block 
slots.  This  process  was  repeated  until  each  scenario  listing  was  complete,  and  the  scenario 
model  was  finished. 

This  process  taught  the  modeler  two  valuable  lessons: 

•  First,  while  the  changing  of  block  names  was  fairly  straightforward,  the  coloring  of  each 
block  was  a  more  difficult  matter.  This  was  due  to  the  fact  that,  in  order  to  maintain 
consistency,  the  exact  same  color  had  to  be  used  for  each  similar  block.  Since  the  color 
palette  is  not  an  exact  way  to  select  a  color,  the  modeler  was  left  with  the  choice  of 
picking  a  color  from  the  palette  and  seeing  if  it  matched  the  color  of  the  related  blocks, 
or  entering  the  specific  RGB  values  for  the  desired  color;  either  process  is  time 
intensive. 

•  The  second  lesson  learned  attests  to  the  value  of  modeling.  The  same  modeler  built  the 
Scenario  Generator  model  and  the  News  Agency  Ontology  model;  therefore,  based  on 
the  modeler’s  familiarity  with  both  works,  the  same  language  should  have  been  used  for 
the  scenario  models  as  was  used  in  the  ontology  model.  This  is  the  case  in  all  scenario 
listings  except  for  one,  the  object  of  the  twelfth  scenario  listing  in  Scenario  l,  News 
Story.  News  Story  was  not  a  term  used  in  the  News  Agency  Ontology,  but  was  the  best 
term  to  describe  the  concept  that  the  modeler  was  trying  to  convey,  that  personnel  need 
to  be  dispatched  to  a  specific  location  to  gather  information.  This  omission  from  the 
News  Agency  Ontology  was  realized  when  the  Scenario  Generator  outputs  were  being 
examined  by  the  project  team.  This  discovery,  and  ensuing  discussion,  solidifies  the 
value  of  using  a  model  to  reason  about  a  problem  and  to  communicate  with  others.  Had 
this  communication  of  the  model  never  occurred,  this  omission  of  critical  terminology 
would  probably  not  have  been  realized.  The  project  team’s  consensus  was  that  the  News 
Story  is  a  generalized  term  for  New,  Existing,  Continuing  and  Background  Information, 
but  the  correction  of  the  News  Agency  Ontology  was  withheld  to  allow  discussion  of  this 
situation  in  the  report. 

The  same  process  used  for  Scenario  l  was  employed  in  the  construction  of  Scenarios  2-5.  Due 
to  space  concerns,  the  diagrams  of  Scenarios  2  and  3  can  be  seen  below  (Figures  23  and  24 
respectively). 
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Scenario  2:  A  news  agency  assigning  a  reporter  to  independently  corroborate  a 
news  source 

Scenario  1  Scenario  2  Scenario  3  Scenario  4  Scenario  5 


Time  Environmental 

Corroborate  Source  Identity  Operational  Constraints 

Identity  Operational  Objectives  Operational  Policies 

Capabilities 

Identify  Personnel 

Dispatch  Personnel 

Mine  <  Information  Archives 

Identity  Information  Sources 

Communicate  Contact 

Interview  Contact 

Reliability  Accuracy 

Evaluate  Information  Source 

Gather  <  Information 

Statistical  Accuracy 

Analyze  Information 

Investigate  Information 

Fact  Check  Information 


Figure  23:  Scenario  2  -  Corroborate  Source 
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Scenario  3:  A  reporter  works  to  recruit  a  new  contact  for  a  story. 


Scenario  1 

Scenario  2 

Scenario  3 

Scenario  4  Scenario  5 

Time  Environmental 

Recruit  New  C ontact  Identity  Operational  Constraints 


Identity  Operational  Objectives  Operational  Policies 


Location  Type 

Identify  Contact 


Capabilities 

Identify  Personnel 


Availability  Location 

Evaluate  Contact 


Communicate  Contact 


Type  Type  Type 

Dispatch  TTransportVehicle  Equipment  -  Personnel 


Interview  Contact 


Reliability  Accuracy 

Evaluate  Contact 


Figure  24:  Recruit  New  Contact 

Because  the  Scenario  Generator  was  built  as  a  simple  prototype,  some  desired  features  and 
capabilities  have  been  withheld  to  allow  for  ease  of  development  and  flexibility  of  use.  Based  on 
analysis  of  the  current  implementation,  additional  functionality  that  could  be  added  to  enhance 
effectiveness  includes: 


•  Instead  of  the  blocks  being  blank,  some  would  be  pre-named  and  pre-colored  to  match 
some  of  the  discovered  scenario  primitives.  There  would  be  a  block  palette  on  the  left 
side  that  would  display  high  level  categories  of  building  blocks.  Clicking  on  such  a  high 
level  label  would  expand  the  category  to  show  pre-named,  pre-colored  blocks  that  were 
to  be  most  commonly  used  primitives.  There  would  also  be  a  set  of  blank  blocks  that 
would  allow  the  modeler  to  name  and  color  blocks  to  their  specification 

•  The  Choose  Piece  Color  window  would  hold  past  color  selections  for  future  reference. 
Although  the  Choose  Piece  Color  window  in  Figure  21  contains  an  area  for  recent  colors, 
the  set  of  recent  colors  resets  when  a  new  block  is  selected.  By  allowing  the  Scenario 
Generator  to  store  colors  selected  across  many  blocks,  the  time  and  effort  needed  to 
exactly  match  each  block’s  color  would  be  greatly  reduced. 

•  A  specific  workspace  would  be  saved  for  future  work.  As  it  exists  now,  when  the  tool  is 
closed,  all  work  is  lost.  Clearly  this  is  not  appropriate  for  a  concept  engineering  effort, 
where  modeling  is  an  iterative  process  that  will  frequently  be  spread  across  a  long 
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period  of  time.  To  be  useful  to  a  modeler,  a  further  developed  Scenario  Generator  would 
need  to  allow  for  saving  and  reopening  of  model  files. 


5.3  Applicability  to  Other  CONOPS/Domains 

The  notion  of  transporting  objects  between  locations,  information  gathering,  and 
communications  are  core  to  many  actions  that  might  be  modeled.  For  instance,  when  looking 
at  a  military  mission  of  close  air  support,  objects  are  moving  from  one  location  to  another,  at 
specific  times,  and  there  is  a  high  degree  of  communication  and  collaboration  necessary. 

Joint  Publication  3-09.3,  dated  8  July  2009,  provides  joint  doctrine  for  executing  close  air 
support.  Chapter  V  Execution  contains  scenarios  for  each  types  of  close  air  support.  The 
appendices  contain  glossaries,  check  lists,  etc.  One  approach  to  adapting  to  other  domains 
would  be  to  extend  the  libraries  of  primitives  to  include  domain  specific  primitives.  A  second 
approach  would  be  to  create  a  table  of  equivalencies  for  domain  specific  language.  In  this  way, 
if  an  equivalent  term  is  available  depending  on  the  domain,  it  is  used.  Otherwise,  the  scenario 
generator  would  use  the  core  primitives.  Yet  another  approach  is  to  create  a  unique  set  of 
libraries  for  each  domain.  Further  research  and  design  are  necessary  to  determine  the  best 
approach,  but  it  is  sufficient  at  this  point  to  know  there  are  a  number  of  alternatives  to  enable 
this  concept  for  multiple  domains. 
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6  System  Architecture 


When  one  considers  what  is  needed  for  a  concept  engineering  system  (CES),  the  high  level 
architecture  is  not  overly  complicated.  There  must  be  a  highly  collaborative  user  interface,  the 
capability  to  create  scenarios  and  the  complete  CONOPS,  a  repository  for  reusable  elements  for 
use  in  new  scenarios,  the  ability  to  add  to,  or  extend  the  primitives,  and  an  execution  engine  to 
put  the  scenarios  in  motion.  Another  capability  that  needs  to  be  included  is  the  ability  to 
interface  with  other  systems/tools  to  exchange  data.  This  must  be  bidirectional  in  that  the  CES 
must  be  able  to  accept  data  from  another  tool  as  well  as  export  data  to  a  downstream  tool. 

6.1  High  Level  System  Architecture 

Figure  25  represents  this  logical  architecture.  In  this  diagram,  The  CES  is  comprised  of  the 
following  major  components: 

1.  External  system  interface 

2.  User  collaboration  interface 

3.  Scenario  generator 

4.  Execution  engine 


Figure  25:  Concept  Engineering  System  (CES)  Logical  Architecture 
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The  user  collaboration  interface  has  several  major  components.  The  first  is  the  concept 
creation  station.  This  may  be  a  single  workstation  such  as  a  horizontally  mounted  computing 
table,  or  may  be  a  collection  of  multiple  workstations.  The  second  component  is  the  concept 
playback  component.  This  is  probably  a  large  screen  or  projection  unit  where  operational 
concepts  that  have  been  completed  can  be  viewed  by  a  larger  group.  Finally,  the  user 
collaboration  interface  must  have  a  report  generation  capability.  These  reports  may  be  textual, 
but  they  may  also  be  a  recording  in  some  video  format  that  can  be  burned  on  a  DVD  for 
sharing. 

The  Scenario  generator  component  must  have  access  to  libraries  of  elements  and  fragments 
(collections  of  elements  that  have  already  been  assembled)  that  can  be  imported  into  a  new 
scenario.  It  must  also  have  the  ability  to  add  new  domain  specific  primitives  to  the  system  for 
future  use. 

Potential  connectivity  for  the  major  components  of  the  CES  high-level  architecture  is 
represented  in  Figure  26. 


Figure  26:  CES  Logical  Architecture  Connectivity 


6.2  Potential  CES  Components 

Phase  1  of  this  research  identified  several  potential  software  tools  that  might  be  part  of  the 
solution  space.  Upon  further  research  carried  out  in  this  phase,  some  of  those  options  still  are 
promising.  And,  others  have  also  emerged  as  the  requirements  are  better  understood.  The  CES 
physical  architecture  is  comprised  of  the  end-user  interaction  devices,  scenario  playback 
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environments  and  the  integration  infrastructure  needed  to  enable  the  CES  logical  architecture, 
shown  in  Figures  25  and  26. 

The  primary  criteria  in  selecting  an  appropriate  component  for  a  specific  concept  engineering 
application  are  the  necessary  cognitive  exchange  bandwidth,  which  is  a  function  of  how  much 
information/knowledge  needs  to  exchanged  between  the  end-users;  available  time  and  budget, 
which  restrict  the  complexity  of  options  used;  and  the  reusability  of  the  infrastructure,  which 
determines  optimal  initial  setup  investment  limits.  While  most  concept  engineering 
frameworks  address  the  graphical  or  visual  aspects  of  end-user  interaction,  the  next-generation 
infrastructures  need  to  focus  on  maximizing  the  cognitive  exchange  bandwidth  between 
concept  developers  at  an  appropriate  time/cost  investment  level. 

6.2.1  Programming  Languages 

While  there  are  a  large  number  of  programming  languages  available  today,  they  fall  into  3 
categories  shown  in  Table  2.  This  section  is  not  intended  to  be  a  computer  science  tutorial,  and 
it  does  not  even  begin  to  scratch  the  surface  of  the  topic. 


Table  2:  Example  Categories  of  Programming  Languages 


Procedural 

Object  Oriented 

Specialty 

Fortran 

Java 

Lisp 

C 

C++ 

Prolog 

Cobol 

Python 

Aspect  oriented  languages 

Each  of  the  above  languages  offer  advantages  and  disadvantages.  When  high  performance  is 
necessary,  C/C++  is  used,  though  that  brings  a  set  of  challenges  such  as  managing  pointers  and 
preventing  memory  leaks. 

As  was  discussed  earlier  in  this  report,  the  taxonomy  resulted  in  a  collection  of  primitives,  the 
vast  majority  of  which  were  objects.  Therefore,  while  Blender  (which  is  discussed  in  Section 
6.2)  is  written  in  C/C++  for  performance,  it  makes  sense  that  Blender  also  allows  users  to  use 
Python  (an  Object  Oriented  scripting  language)  to  manipulate  the  environment. 

The  Scenario  Generator  Demo  was  written  in  Java.  Most  computers  today  have  either  the  Java 
Runtime  Engine  (JRE)  or  the  Java  Development  Kit  (JDK)  installed  due  to  the  pervasiveness 
of  Java  use  on  the  Web. 

Execution  engines  such  as  Open  Simulator  (which  will  also  be  discussed  in  Section  6.2)  are 
written  in  C#,  which  is  a  Microsoft  language  that  was  influenced  predominately  by  Java. 

There  is  no  clear  recommendation  for  a  single  programming  language  for  the  CES.  Each  open 
source  programming  initiative  has  its  language  of  choice.  Therefore,  it  will  be  dependent  on  the 
amount  of  open  source  software  that  is  incorporated  as  to  which  programming  language  is  best 
for  the  new  portions  of  CES.  What  may  become  more  important  will  be  the  application 
programming  interfaces  (API)  options  provided  by  each  software  program  considered  for 
incorporation  into  the  CES. 
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6.2.2  Concept  Creation  Stations 


Concept  Creation  Station  is  the  element  of  the  CES  physical  infrastructure  that  allows  the  end- 
users  to  interact  with  the  system  and  each  other.  Concepts  of  operation  are  created  by  dragging 
and  dropping  CONOPS  primitives  onto  a  screen  and  connecting  via  operations  from  a  mouse. 
Several  distributed  Concept  Creation  Stations  could  be  linked  to  enable  distributed  teams  to 
collaborate.  Following  are  descriptions  of  some  of  the  potential  technologies  that  can  be  used 
for  Concept  Creation  Stations. 

Sift  Table  Smart  Blocks 

SiftTable  Smart  Blocks  are  physical  blocks  that  have  functionality  associated  with  them. 
Additionally,  these  smart  blocks  can  also  be  placed  with  others  to  form  groups  that  have  the 
composite  functionality.  Smart  blocks  may  be  used  as  primitives  to  create  concepts  of 
operations  by  placing  them  in  a  specific  logical  arrangement.  The  smart  blocks  concept  is 
attractive  from  the  dynamic  composability  perspective.  However,  the  physical  blocks 
themselves  are  not  very  convenient  to  create  large  systems  or  for  virtual  teams.  Implementing 
the  Smart  Block  concept  within  the  Surface  Computing  hardware  environment  (and  extending 
that  to  a  networked  setting)  offers  a  scalable  solution. 

Surface  Computing 

Surface  Computing  allows  a  number  of  people  to  simultaneously  interact  with  graphical 
building  blocks  to  engineer  a  concept.  In  addition,  this  concept  could  be  extended  to  multiple 
sites  and  distributed  collaboration  with  the  networking  of  a  number  of  tables.  Templates,  or 
patterns,  could  be  used  to  quickly  create  new  combinations  or  variants  of  existing  scenarios. 
Tapping  the  blocks  could  be  used  to  dive  deeper  into  the  hierarchy,  squeezing  could  allow  you 
to  pop  up  in  the  hierarchy.  The  two  scenarios  below  give  an  idea  of  the  use  of  surface 
computing  in  future  CES  (note:  both  the  cases  use  the  Microsoft  Surface  table.  The  first  is 
related  to  the  NEO  rescue  problem,  and  the  second  demos  the  control  of  a  real-time  dynamic 
system): 

1.  http://mrrichie.spaces.live.com/blog/cnslDD16C3F34F4D913EI3112.entry 

2.  http://kiwigis.bl0gsp0t.c0m/2009/06/p0lice-dispatcher-0n-micr0s0ft-surface.html 

HubNet 

HubNet  is  a  network  of  handheld  devices  and  up-  front  computer,  developed  for  classroom 
participation  application  (http://ccl.northwestern.edu/netlogo/hubnet.html).  HubNet  is 
designed  specifically  with  the  goal  of  creating  a  fully  participatory,  networked,  classroom 
learning  space.  The  core  features  of  this  space  include: 

1.  A  collection  of  hand-held  computing  devices  with  significant  resident 
functionality  (e.g.,  a  programmable  graphing  calculator); 

2.  A  network  layer  of  flexible  communication  protocols  including  the  ability  to 
upload  and  download  data  sets,  upload  and  download  program  (e.g.,  applets), 
monitor  key-presses  at  the  hand-held  level,  support  real-time  interaction  as  in 
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network  computer  games,  and  form  collaborative  groups  of  various  sizes  (e.g., 
peer  to  peer,  small  groups,  and  whole  class  modes); 

3.  Development  tools  for  programming  individual  devices  and  the  network; 

4.  Public  display  capability  (e.g.,  a  projection  system); 

5.  An  up-front  computer  (the  "hub")  capable  of  addressing  the  network  of 
handhelds  and  running  systems  modeling  programs  (e.g.,  NetLogo,  STELLA)  as 
well  as  core  analytic  tools  (spread  sheets,  function  graphs,  etc.); 

6.  An  ability  to  be  addressed  using  open  standards  and  protocols; 

7.  A  cross-platform  implementation  (using  HTML,  Java,  etc.); 

8.  An  ability  to  connect  and  interact  with  other  layers  of  network  'up'  from  the 
classroom,  including  various  school-  based  LANs,  district  WANs,  and  the 
Internet. 

HubNet  devices  can  be  used  by  concept  developers  to  choose  primitives,  create  logical 
arrangements  and  interact  with  other  developers. 

Multipoint 

Multipoint  technology  allows  the  use  of  2-250  active  mice  on  a  single  computer.  Using  mice  as 
a  primary  interaction  interface  is  certainly  a  cost  effective,  easy  and  intuitive  option  for  most 
users.  An  SDK  can  be  downloaded  to  assist  in  the  development  process.  Collaborators  can  sit 
around  a  table,  each  with  their  own  wireless  mouse,  participating  in  a  concept  engineering 
exercise  on  a  shared  large  screen.  Multipoint  can  be  deployed  in  any  modern  conference  room 
with  the  addition  of  wireless  mice. 

6.2.3  Primitive  Creation 

Primitive  creation  was  explored  as  part  of  this  phase.  It  is  recommended  that  the  lego-like 
interface  continue  to  be  expanded  so  it  generates  XML  that  can  be  utilized  by  the  scenario 
generator  in  conjunction  with  libraries  and  other  scenario  fragments. 

6.2.4  Scenario  Generator  and  Concept  Playback 

Scenario  generator  allows  the  end-users  to  create  a  complex  concept  using  primitives, 
represent  all  the  interactions,  capture  the  rationale  and  allow  the  session  to  be  played  back  for 
analysis  and  reuse. 

Typically,  3D  presentations  of  information  can  be  very  informative  and  impressive.  The 
scenario  generator  and  concept  playback  can  provide  a  number  of  different  views,  both  2D  and 
3D.  The  2D  views  can  be  of  an  entire  system  or  of  a  particular  layer  in  a  system  hierarchy. 
Moving  to  a  3D  view,  various  layers  in  the  hierarchy  can  be  presented.  What  is  represented  in 
a  particular  layer  can  be  selected  by  each  user  which  enables  them  to  view  the  system  from  a 
number  of  different  vantage  points.  With  multiple  screens,  a  number  of  participants  can  each 
interact  with,  build  and  modify  the  system  simultaneously,  each  from  their  own  perspective.  A 
global  view  could  be  presented  to  all  on  the  large  screen  on  the  wall,  while  each  participant 
interacts  on  their  own  workstation.  Finally,  each  user  can  have  personalized  information 
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relevant  for  the  stakeholders  that  he/she  represents  on  his/her  view.  This  information  can  be 
customized  to  provide  each  user  with  what  they  need  to  see,  while  the  information  pertaining 
to  the  global  good  can  be  viewed  on  a  global  screen.  It  should  be  noted  that  both  the  CONOPS 
structure  and  the  simulation  outcomes  can  be  personalized  for  each  of  the  viewers.  With  a 
package  of  scenarios,  the  end-users  quickly  build  up  the  complexity  of  the  CONOPS  to  address 
ever  increasingly  challenging  requirements.  Following  are  some  descriptions  of  the  potential 
technologies  that  can  be  used  for  Scenario  Generator  and  Concept  Playback  modules. 

E-Wall 

EWall  introduces  a  computational  framework  for  the  formation  of  ideas  in  a  collective 
brainstorming  process.  EWall  is  a  web-based  environment  that  allows  users  to  collect,  organize 
and  view  graphical  and  contextual  information.  It  introduces  new  methodologies  for 
brainstorming,  supports  the  negotiation  process  among  multiple  users  and  provides 
mechanisms  to  arrange  data  in  various  ways.  The  objective  is  to  reduce  the  necessary  amount 
of  verbal  communication  during  a  brainstorming  process  in  order  to  improve  efficiency,  allow 
more  people  to  collaborate  and  encourage  asynchronous  and  remote  participation. 

Blender 

Blender  is  an  open  source  3D  content  creation  suite,  available  for  all  major  operating  systems 
under  the  GNU  General  Public  License.  It  provides  the  ability  to  create  professional  quality 
graphics  that  would  make  a  CONOPS  come  to  life.  Examples  of  artwork  created  with  Blender 
are  shown  below.  There  is  a  3D  Open  Movie  Project:  SINTEL  which  will  be  a  short  movie.  The 
graphics  are  all  created  using  Blender.  The  trailer  can  be  found  at: 
http://www.youtube.com/watch?v=HOfdboHvshg. 

The  quality  of  the  tool  has  been  compared  to  that  of  Maya,  which  is  a  commercial  tool.  The 
graphics  below  were  generated  using  Blender  and  can  be  found  on  the  Blender  website. 


OpenSimulator 

OpenSimulator  is  a  3D  grid-based  application  server.  It  can  be  used  to  create  a  virtual 
environment  (or  world)  which  can  be  accessed  through  a  variety  of  clients,  on  multiple 
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protocols.  There  are  versions  for  Windows,  Linux  and  Apple.  The  software  is  built  as  loadable 
modules  for  custom  configurations,  and  is  released  under  a  BSD  License. 

Out  of  the  box,  OpenSimulator  can  be  used  to  simulate  a  virtual  environment  similar  to  Second 
Life™  (including  client  compatibility).  Other  environments,  protocols  and  features  are 
supported  via  add  on  modules.  OpenSimulator  is  still  considered  alpha  software,  but  seems  to 
have  a  large  and  growing  following.  The  following  graphics  can  be  found  in  OpenSimulator 
grids  that  are  openly  available. 


Second  Life 

Second  Life  (SL)  is  a  virtual  world  developed  by  Linden  Lab  that  is  accessible  on  the  Internet 
(http://  http://secondlife.com).  A  free  client  program  called  the  Viewer  enables  its  users, 
called  Residents,  to  interact  with  each  other  through  avatars.  Residents  can  explore,  meet  other 
residents,  socialize,  participate  in  individual  and  group  activities,  and  create  and  trade  virtual 
property  and  services  with  one  another,  or  travel  throughout  the  world  (which  residents  refer 
to  as  "the  grid"). 

Built  into  the  software  is  a  three-dimensional  modeling  tool  based  around  simple  geometric 
shapes  that  allows  a  resident  to  build  virtual  objects.  This  can  be  used  in  combination  with  the 
Linden  Scripting  Language  which  can  be  used  to  add  functionality  to  objects.  More  complex 
three-dimensional  sculpted  prims  (colloquially  known  as  sculpties),  textures  for  clothing  or 
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other  objects,  and  animations  and  gestures  can  be  created  using  external  software.  The  Second 
Life  Terms  of  Service  ensure  that  users  retain  copyright  for  any  content  they  create,  and  the 

server  and  client  provide  simple  digital  rights 
management  functions. 

This  thoroughly  modern  Peugeot  Island, 
featuring  Design,  will  make  it  possible  for 
"Second  Lifers"  to  discover  in  3D  the  concept 
car  Flux,  winner  of  the  fourth  Peugeot  Design 
Contest,  a  full-scale  model  of  which  is 
presented  on  the  Peugeot  stand  at  the 
Frankfurt  Motor  Show. 

Khronos  Projector 

The  goal  of  the  Khronos  Projector  is  to  go 
beyond  these  forms  of  exclusive  temporal 
control,  by  giving  the  user  an  entirely  new 
dimension  to  play  with:  by  touching  the 
projection  screen,  the  user  is  able  to  send  parts  of  the  image  forward  or  backwards  in  time.  By 
actually  touching  a  deformable  projection  screen,  shaking  it 
or  curling  it,  separate  "islands  of  time"  as  well  as  "temporal 
waves"  are  created  within  the  visible  frame.  This  is  done  by 
interactively  reshaping  a  two-dimensional  spatio-temporal 
surface  that  "cuts"  the  spatio-temporal  volume  of  data 
generated  by  a  movie.  The  Khronos  projector  unties  time  and 
space  in  a  pre-recorded  movie  sequence,  opening  the  door 
for  an  infinite  number  of  interactive  visualizations.  Using  the 
Khronos  projector,  event's  causality  become  relative  to  the 
spatial  path  we  decide  to  walk  on  the  image,  allowing  for  a 
multiple  interpretation  of  the  recorded  facts.  In  this  sense, 
the  Khronos  projector  can  be  seen  as  an  exploratory  interface 

that  transforms  a  movie  sequence  into  a  spatio-temporal  sculpture  for  people  to  explore  at 
their  own  pace  and  will.  http://www.k2.t.u-tokyo. ac.jp/members/alvaro/Khronos/ 
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7  Recommendations  for  Future  Research 


In  this  section,  we  present  our  recommendations  for  future  research.  We  begin  by 
summarizing  how  the  first  two  phases  of  this  project  have  positioned  us  to  undertake  the 
requisite  work. 

In  Phase  l,  we  developed  the  Agile  Concept  Engineering  Phases  (see  Figure  27)  and  highlighted 
how  the  mental  model  content,  primitive,  and  tool  requirements  would  be  a  function  of  these 
various  phases.  Moreover,  we  identified  cognitive  task  analysis  and  content  decomposition  as 
two  analysis  techniques  that  would  help  us  to  achieve  our  objectives  and  presented  various 
potential  software  and  hardware  products  that  might  be  appropriate  for  the  tool  we  were 
envisioning. 


Figure  27:  Activities  in  Phases  1  and  2 


In  the  current  phase,  we  have  devised  methodologies  for  identifying  mental  model  content  and 
primitives  through  cognitive  task  analysis  and  content  decomposition,  developed  the 
framework  and  initial  primitive  taxonomy,  as  well  as  establishing  a  high-level  architecture  for 
the  concept  engineering  system  (i.e.,  the  tool). 

Based  on  the  research  conducted  in  the  first  two  phases,  we  will  now  present  our  roadmap  for 
developing  a  prototype  concept  engineering  system  that  is  applicable  across  a  wide  array  of 
application  domains. 
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7.1  Concept  Engineering  System  Development 

The  research  that  has  been  performed  up  to  this  point  served  to  lay  the  framework  for 
understanding  the  requirements  for  a  concept  engineering  system  (CES).  That  framework  is 
shown  in  Figure  28,  in  which  the  CES  must  support  the  agile  concept  engineering  phases 
identified  in  phase  1,  and  support  the  creation  and  management  of  both  the  core  and  domain- 
driven  mental  model  content  and  primitives. 


Concept  Engineering  System 
for 


Agile  CONOPS  Development 


r 


r 


Domain-Driven  Mental  Model  Content  &  Primitives 


Core 

Mental  Model 
Content  & 
Primitives 


Figure  28:  Inputs  to  Concept  Engineering  System 


7.1.1  Integration 

The  first  two  phases  of  this  research  identified  a  number  of  robust  products  that  are  used  in 
other  domains  to  develop  high  quality  models  and  simulations  that  are  rich  in  capability  and 
fidelity.  Section  6.1  detailed  a  logical  architecture  for  the  CES.  Much  of  the  next  phase  will 
require  working  out  the  details  of  modifying  and  integrating  existing  software,  and  designing/ 
building  the  “glue”  that  will  create  a  cohesive  and  easy  to  use  collaborative  concept  engineering 
system. 

7.1.2  CES  Development 

Scrum,  as  it  is  applied  to  software  development,  is  shown  in  Figure  29  can  be  modified  to  be 
applicable  in  a  systems  engineering  environment  (Crowe  and  Cloutier,  2009).  The  CES 
development  will  be  performed  using  Scrum.  It  is  recommended  that  the  opportunity  be  taken 
to  identify  and  capture  relevant  metrics  relating  to  the  use  of  Scrum  in  a  research  & 
development  (R&D)  environment. 

One  modification  to  the  typical  application  of  Scrum  that  may  need  to  be  adjusted  is  the 
timeframes.  When  Scrum  is  used  in  a  software  development  organization,  Sprints  can  be  1 
week  to  1  month  in  duration.  In  the  university  development  environment,  it  may  be  necessary 
to  use  the  longer  period  -  one  month,  per  sprint.  The  shorter  meeting  -  the  Scrum,  may  be 
better  suited  to  a  once  weekly  meeting  of  the  collaborators. 
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Scrum:  1 5  minute  meeting 
Team  members  respond  to  basic 
questions: 

1 )  What  did  you  do  since  last 
Scrum  meeting 

2)  Do  you  have  any  obstacles? 

3)  Whatwillyoudo  beforethe  next 
meeting? 


Product  Backlog  Sprint  Backlog  Sprint  Working  increment 

of  the  software 

Prioritized  features  Featuresassigned 

desired  by  customer  to  Sprint 

Backlog  items  added 
by  team  members 


Figure  29:  Software  Scrum  Approach 


7.2  Further  Scenario  Development 

This  phase  of  the  research  looked  at  intelligence  gathering  tasks  in  both  the  military  and  news 
domains.  We  applied  a  set  of  techniques  including  cognitive  task  analysis  and  decomposition 
to  identify  the  basic  functions  required  to  complete  these  tasks.  Further,  we  mined  the 
primitives  into  a  taxonomy  of  reusable 
terms  and  objects.  The  result  of  these 
research  activities  is  a  set  of 
procedures  for  identifying  mental 
model  content  and  taxonomies  that 
can  be  applied  to  other  domains. 

We  recommend  that  a  wide  range  of 
additional  domains  be  investigated 
during  the  next  research  phase  of  this 
project  (see  Figure  30).  Once  the 
target  domains  are  identified,  the  team 
will  identify  additional  operational 
scenarios  for  cognitive  task  analysis, 
decomposition  and  modeling. 

Additionally,  we  will  apply  the 
aforementioned  methodologies  to  build  the  libraries  of  mental  model  content  and  primitives. 


Core 

Mental  Model 
Content  & 
Primitives 


Domain-Driven  Mental  Model  Content  &  Primitives 


Figure  30:  Process  for  Adding  Libraries 
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7.3  Experimentation 


Experimentation  results  will  provide  feedback  to  the  concept  engineering  system  development 
process  throughout  the  next  phase  of  this  project  (see  Figure  31).  Specifically,  as  components 
and  versions  of  the  prototype  are  developed,  experiments  will  be  conducted  to  test  those 
elements.  The  experimental  design  will  be  driven  by  research  questions  focused  on  component 

functionality,  human-computer 
interfaces,  cognitive  load,  data 
import/exportability,  etc.  Results 
from  the  experiments  will  be  used  to 
inform  the  next  generation 
prototype  of  the  concept  engineering 
system. 


Concept  Engineering  System 
for 

Agile  CONOPS  Development 


Core 

Mental  Model 
Content  & 
Primitives 


f 


1 


Scenario  selection  will  be  a  critical 
aspect  of  experimentation.  The 
scenario(s)  that  subjects  will  be 
asked  to  complete  must  balance 
realism/relevance  for  the  project 
sponsors  with  experimental 
requirements.  The  intelligence 
gathering  tasks  that  have  been  the 
focus  of  our  efforts  in  the  current 
phase  may  prove  to  be  difficult  for 
student  subjects  to  complete  due  to 
their  limited  background  knowledge 
Figure  si!  Iterative  Approaeh  of  the  scenario  domain.  Therefore. 

the  research  team  will  work  with  the  sponsors  to  identify  tasks  that  meet  the  criteria  for  an 
experimental  testbed,  while  at  the  same  time  are  relevant  to  the  sponsors’  domains  of  interest. 
One  potential  scenario  for  these  purposes  is  the  “Special  Operations  Reconnaissance  (SOR) 
Scenario”  developed  at  the  Naval  Air  Warfare  Center  Aircraft  Division  (Warner,  Burkman,  and 
Biron,  2008). 


Experimentation 


7.4  Proposed  Project  Plan 

The  next  phase  of  research  will  be  entail  significant  amount  of  effort  and  it  will  be  necessary  to 
expand  the  size  and  expertise  of  the  team  and  the  work  that  must  be  performed.  This  will 
require  establishing  a  collaborative  development  environment  and  testbed.  An  OpenSimulator 
grid  will  be  established,  managed,  and  maintained  to  support  the  agile  development  of  the 
CES.  The  first  year  will  deliver  a  prototype,  and  experiments  to  measure  the  effectiveness  of  the 
prototype  in  satisfying  the  agile  CONOPS  process.  The  results  of  those  experiments  will 
become  the  product  backlog  which  becomes  the  sprint  tasking  for  the  second  year  (Figure  29). 

Table  3  represents  the  team’s  recommendation  for  follow-on  work  on  this  task. 
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Table  3:  Concept  Engineering  Research  Plan 


Year  Deliverables 


Year  1 

Concept  Engineering  System  (CES)  Architecture 

1st  CES  Prototype 

Concept  Engineering  Process 

Prototype  experiments  using  students 
(undergraduate  and  industry  graduate) 

Requirements  for  2nd  iteration 

Year  2 

2nd  CES  prototype  iteration 

2nd  experiments,  with  customer  participants 
Requirements  for  3rd  iteration 

3rd  CES  prototype 

Year  3 

Field  test  CES 

Requirements  for  4th  iteration 

4th  CES  prototype 
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Appendix  A:  NEO  Intelligence  Gathering  Scenario 


Noncombatant  Evacuation  Operation: 
Intelligence  Gathering  Scenario 

Intelligence  Expert  Briefing 

You  are  an  Intelligence  Officer  for  an  organization  within  the  U.S.  Department  of  Defense  in 
the  U.S.  Pacific  Command.  As  a  military  intelligence  officer,  your  responsibilities  may  include 
collecting  information  about  the  operational  environment;  hostile,  friendly  and  neutral  forces; 
the  civilian  population  in  an  area  of  combat  operations;  and  other,  broader  areas  of  interest. 
This  information  may  be  gathered  from  personnel  on  the  ground  in  the  area  of  interest,  the 
Internet,  drone  and  satellite  data,  other  agencies  or  personnel,  etc. 

It  is  currently  2:05  a.m.  on  January  15.  You  just  finished  reading  an  alert  that  came  across  your 
handheld  device  at  2:00  a.m.  from  the  American  Red  Cross.  Three  of  their  aid  workers  are 
trapped  on  a  remote  island  in  the  South  Pacific  and  are  caught  in  the  middle  of  guerilla 
warfare. 

You  immediately  contacted  a  weapons  and  an  environmental  expert  to  meet  with  you  about 
this  situation.  Your  first  task  will  be  to  ascertain  what  information  is  necessary  to  create  a 
mission  plan  for  extracting  the  Red  Cross  personnel.  Since  contacting  them,  you  have  had  a 
brief  communication  with  your  contact  on  the  ground  who  is  working  with  the  local  military  to 
fight  the  guerillas.  The  aid  workers  escaped  the  village  and  are  somewhat  safe  in  one  room  of  a 
church.  They  have  no  source  of  water  except  rainwater.  At  least  one  worker  is  in  need  of 
medical  attention.  You  were  cutoff  before  you  could  get  any  more  information  from  him. 

When  you  and  the  others  have  finished  reading  your  briefings,  your  meeting  will  begin.  Your 
deliverable  will  be  a  comprehensive  set  of  questions  about  the  information  and  intelligence 
needed  to  plan  the  extraction  mission.  The  questions  should  be  organized  by  specialty  (e.g., 
intelligence)  and  subject  matter  (e.g.,  weather). 

As  you  work  together  to  complete  the  deliverable,  be  sure  that  you  consider  your  expert 
perspective.  You  may  need  particular  information  that  one  of  the  other  team  members  can 
collect  for  you  or  you  may  be  in  a  position  to  gather  information  that  will  help  one  of  them. 
Also,  you  must  be  as  specific  as  possible.  For  example,  asking  “What  is  the  weather  forecast?”  is 
too  vague.  You  need  to  clarity  exactly  what  information  about  the  weather  you  may  need  by 
including  questions  such  as,  “What  is  the  chance  of  rain?  What  is  the  timeframe?” 
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Noncombatant  Evacuation  Operation: 
Intelligence  Gathering  Scenario 

Weapons  Expert  Briefing 

You  are  a  newly  hired  Weapons  Expert  for  an  organization  within  the  U.S.  Department  of 
Defense  in  the  U.S.  Pacific  Command.  Your  responsibilities  include  providing  information 
about  the  military  capabilities  available  for  missions,  including  transport,  personnel,  and 
weapons.  You  have  some  basic  knowledge,  but  are  still  learning.  You  have  colleagues  at  other 
Commands  that  can  provide  you  with  the  information  you  do  not  know. 

It  is  currently  2:05  a.m.  on  January  15.  An  Intelligence  Officer  in  your  Command  has  just 
informed  you  that  three  American  Red  Cross  workers  are  trapped  on  the  remote  South  Pacific 
island  of  Drapo  and  are  caught  in  the  middle  of  guerilla  warfare.  You  are  to  attend  a  meeting 
with  the  Intelligence  Officer  and  others  to  ascertain  what  information  is  necessary  to  create  a 
mission  plan  for  extracting  the  Red  Cross  personnel. 

Quickly,  you  ran  reports  to  identify  the  military  assets  in  the  region.  You  prepared  the  attached 
reports  about  the  Navy  SEALS,  Army  Special  Forces,  and  Marine  assets  available,  and  included 
the  limited  information  you  know  about  their  capabilities.  You  are  certain  additional,  more 
specific,  information  will  be  needed.  But  have  decided  to  wait  until  the  meeting  to  see  what 
others  deem  necessary. 

When  you  and  the  others  have  finished  reading  your  briefings,  your  meeting  will  begin.  Your 
deliverable  will  be  a  comprehensive  set  of  questions  about  the  information  and  intelligence 
needed  to  plan  the  extraction  mission.  The  questions  should  be  organized  by  specialty  (e.g., 
weapons)  and  subject  matter  (e.g.,  weather). 

As  you  work  together  to  complete  the  deliverable,  be  sure  that  you  consider  your  expert 
perspective.  You  may  need  particular  information  that  one  of  the  other  team  members  can 
collect  for  you  or  you  may  be  in  a  position  to  gather  information  that  will  help  one  of  them. 
Also,  you  must  be  as  specific  as  possible.  For  example,  asking  “What  is  the  weather  forecast?”  is 
too  vague.  You  need  to  clarity  exactly  what  information  about  the  weather  you  may  need  by 
including  questions  such  as,  “What  is  the  chance  of  rain?  What  is  the  timeframe?” 
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Navy  Seals  Assets 

Personnel: 

E  7-man  squad  consisting  of: 

~  l  Corpsman 

~  l  Radioman 

~  l  Heavy  Gunners 

~  2  Riflemen  with  M-i6  rifles 

~  2  Riflemen  with  M-i6  rifles  and  grenade  launchers 

E  Night  vision  capability 

E  Need  local  translator  if  communicating  with  villagers. 

E  Ability  to  be  virtually  undetected 

E  All  Navy  SEALS  are  trained  as  medics. 

E  Navy  SEALS  usually  initiate  covert  operations  from  the  sea.  (Covert  operations  are 
secret,  undetected  operations.) 

E  All  weapons  and  personnel  are  located  on  the  USS  Enterprise. 

E  The  USS  Enterprise  has  full  medical  facilities. 


Transportation  &  Weapons: 


E  Navy  Seahawk  (SH-6o):  A  twin-engine 
helicopter  used  for  anti-submarine 
warfare,  drug  interdiction,  cargo  lifts 
and  special  operations  in  the  day  or 
night  regardless  of  the  weather 
conditions. 

E  Navy  Hornet  (F-i8):  A  one  or  two  seat 
super  sonic  jet  used  for  air-to-air  or  air- 
to  ground  support. 

>  m  — 

E  Zodiac:  A  7-man  inflatable  boat  which  can 
travel  at  speeds  up  to  15  miles/hour. 

^  <e  4 

Army  Special  Forces  Assets 
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Personnel: 


6-man  Special  Forces  Team  consisting  of: 

~  iTeam  Lead 
~  2  Snipers 
~  2  Team  Medics 

~  iTranslator (Speaks  several  languages,  but  NOT  Draponese.) 

#  Highly  trained  in  multiple  languages. 

£  Night  vision  capabilities 
%  Weapons:  M-16,  M-6o,  Grenades 

Army  Special  Forces  usually  initiate  covert  operations  from  the  land.  (Covert  operations 
are  secret,  undetected  operations.) 


Transportation  &  Weapons: 


Abram  Tanks:  Used  for  enemy 
suppression. 

Blackhawk  Helicopter  (UH-6o):  A  twin 
engine  helicopter  used  for  special  ops  in 
the  day  or  night,  regardless  of  the  weather 
conditions. 

#  AH-64  Apache:  A  twin  engine  helicopter 
used  for  enemy  suppression. 

%  C-130  Hercules:  A  large  aircraft  used  for 
troop  and  cargo  transport,  paratrooper 
deployment  and  airborne  refueling. 
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Marine  Assets 

Personnel: 

►  12  Man  Team  that  includes: 

~  2  Medical  Corpsmen 
~  Use  of  M-16,  M-6o,  Grenades 
~  Night  vision  capabilities 

►  Marine  personnel  are  stationed  on  the  USS  Enterprise. 


Transportation  &  Weapons: 


►  AH-i:  An  attack  force  helicopter  used  to 
suppress  enemy  troops  and  provide  air 
support. 

' 

CH-53:  A  cargo/transport  helicopter  used 
in  amphibious  and  shore  operations. 

■u*ii  jp*  1  ^ 

•  •  L  r  Ij'P  '  '--Ji  -  * 
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Noncombatant  Evacuation  Operation: 
Intelligence  Gathering  Scenario 

Environmental  Expert  Briefing 


You  are  an  Environmental  Expert  for  an  organization  within  the  U.S.  Department  of  Defense  in 
the  U.S.  Pacific  Command.  Your  responsibilities  include  gathering  information  about  the 
geography,  climate,  wildlife,  etc.  for  the  locations  and  surrounding  areas  where  missions  will 
be  executed. 

It  is  currently  2:05  a.m.  on  January  15.  An  Intelligence  Officer  in  your  Command  has  just 
informed  you  that  three  American  Red  Cross  workers  are  trapped  on  the  remote  South  Pacific 
island  of  Drapo  and  are  caught  in  the  middle  of  guerilla  warfare.  You  are  to  attend  a  meeting 
with  the  Intelligence  Officer  and  others  to  ascertain  what  information  is  necessary  to  create  a 
mission  plan  for  extracting  the  Red  Cross  personnel.  You  pulled  the  most  recent  aerial  map  of 
Drapo  Island  (attached)  to  take  with  you  to  the  meeting. 

When  you  and  the  others  have  finished  reading  your  briefings,  your  meeting  will  begin.  Your 
deliverable  will  be  a  comprehensive  set  of  questions  about  the  information  and  intelligence 
needed  to  plan  the  extraction  mission.  The  questions  should  be  organized  by  specialty  (e.g., 
environment)  and  subject  matter  (e.g.,  weather). 

As  you  work  together  to  complete  the  deliverable,  be  sure  that  you  consider  your  expert 
perspective.  You  may  need  particular  information  that  one  of  the  other  team  members  can 
collect  for  you  or  you  may  be  in  a  position  to  gather  information  that  will  help  one  of  them. 
Also,  you  must  be  as  specific  as  possible.  For  example,  asking  “What  is  the  weather  forecast?”  is 
too  vague.  You  need  to  clarity  exactly  what  information  about  the  weather  you  may  need  by 
including  questions  such  as,  “What  is  the  chance  of  rain?  What  is  the  timeframe?” 
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The  South  Pacific  Island  of  Drapo 
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Appendix  B:  NEO  Mission  Planning  Scenario 


The  Mission: 

The  time  is  2:00  am,  January  15th.  Three  American  Red  Cross  workers  are  trapped  in  a  church 
basement  on  a  remote  island  in  the  South  Pacific,  caught  in  the  middle  of  guerilla  warfare. 
Your  mission  is  to  rescue  them  within  24  hours. 

The  situation  is  described  in  the  next  few  pages  along  with  the  assets  of  U.S.  forces  available  to 
rescue  the  workers.  Work  together  to  develop  a  course  of  action,  using  ANY  assets  available  to 
you.  The  detailed  plan  can  be  made  up  of  Army,  Marine  or  Navy  Seal  assets  (or  a  combination 
of  the  three),  but  must  include  the  following: 

a  description  of  how  the  rescuers  will  get  to  the  church 

a  description  of  how  they  will  evacuate  the  Red  Cross  workers 

and  a  description  of  how  they  will  return  to  the  Army  base  or  aircraft  carrier. 

Choose  the  best  and  most  efficient  solution  to  rescue  the  workers  safely.  Minimize  damage  to 
the  village  and  villagers,  and  avoid  contact  with  the  enemy  if  possible.  Keep  in  mind,  however, 
the  rules  of  engagement  state  that  any  forces  will  defend  themselves  if  needed. 

Good  Luck! 
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BACKGROUND  INFORMATION: 
Drapo  Island 


Date: 

January  15th  ,  0200  (2:00  AM),  Drapo  Time  Zone 

Location: 

Drapo  Island,  located  750  miles  north  of  Australia,  slightly  northeast  of  the  Solomon  Islands. 
(See  Map  1) 

The  church  where  the  workers  are  trapped  is  located  3  miles  from  the  northern-most  shore  of 
the  island.  Directly  in  front  of  the  church  is  a  dirt  road,  which  leads  to  the  ocean  in  one 
direction  and  into  the  main  village  on  the  island  in  the  other  direction.  The  village  is  1.5  miles 
from  the  church.  The  road  also  passes  homes  and  farms  of  the  natives  on  the  way  into  the 
town.  The  church  has  some  vegetation  around  it,  primarily  coconut  trees  and  some  brush,  but 
no  heavy  forest  or  swampland.  The  land  around  it  is  mostly  cultivated  for  farming. 

A  small  port  is  located  on  the  opposite  end  of  the  island  from  the  church,  and  is  heavily 
guarded  by  the  local  military.  To  reach  the  church  from  the  port,  you  must  cross  an 
uninhabited  volcanic  mountain  pass,  which  is  covered  by  dense  rainforest. 

Because  of  the  coral  reefs,  it  is  impossible  to  bring  large  ships  any  closer  than  1  mile  from  the 
coast.  The  shore  is  only  accessible  by  small  boats.  Significant  amounts  of  coral  can  be  seen 
above  the  surface  of  the  water  at  low  tide. 

There  is  one  paved  highway  around  the  perimeter  of  the  island.  This  road  also  leads  to  the  dirt 
road  where  the  church  is  located.  (See  Maps  2  and  3) 

Size: 

The  island  is  250  miles  wide  by  250  miles  long 

Topography: 

Drapo  Island  is  made  up  of  volcanic  mountains,  atolls  (ring-like  coral  island  and  reef),  some 
swampy  lowlands,  and  rugged  mountainous  terrain  with  some  low  dense  rainforest.  Because 
much  of  the  island  is  made  up  of  volcanic  mountains,  there  is  a  constant  threat  of  volcanic 
activity.  The  terrain  consists  of  coastal  plains,  swampy  lowlands,  a  large  rainforest  and 
mountain  ranges. 

Climate  and  Weather: 

The  climate  on  Drapo  is  tropical  with  rainy  seasons  from  December  through  March  and  May 
through  October.  There  are  also  periodic  tropical  monsoons.  In  January,  there  are  periods  of 
heavy  fog  in  the  morning,  which  burns  off  as  the  day  progresses.  Visibility  is  usually  clear  by 
noon.  There  are  strong  winds  throughout  the  day. 
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Military: 


The  local  military  is  made  up  of  about  too  volunteers  who  support  their  government.  It  has 
recently  been  built  up  in  response  to  increasing  threats  from  rebel  forces.  The  government  has 
limited  military  intelligence  and  limited  analysis  capability.  Drapo  is  in  good  and  peaceful 
standing  with  the  United  States. 

Village: 

The  village  is  home  to  the  government  center,  school,  church,  markets,  and  communication 
center  for  the  island.  Any  weapons  and  military  intelligence  possessed  by  the  island  will  be 
centered  in  the  village.  The  majority  of  the  island’s  homes  are  located  within  l  mile  of  the 
village.  There  are  between  too  and  150  huts  housing  5  to  10  natives  each.  The  village  is  about 
1.5  miles  from  the  church  and  4.5  miles  from  the  northern-most  point  of  the  island. 

The  local  language  is  Draponese.  The  locals  do  NOT  speak  English,  so  if  any  attempt  is  going  to 
be  made  to  communicate  with  the  local  military,  a  translator  will  be  needed. 

Transportation: 

One  main  paved  highway  surrounds  the  perimeter  of  the  island.  (See  Map  2)  Dirt  roads 
connect  the  airport,  homes,  village,  and  the  church  where  Red  Cross  workers  are  trapped. 

There  is  one  main  port,  which  is  used  for  the  export  and  import  of  goods.  Directly  off  the  shore 
of  most  of  the  island  are  atolls,  which  make  it  impossible  for  large  ships  to  come  within  one 
mile  of  the  shore  (with  the  exception  of  the  port). 

There  is  one  main  airport  with  a  paved  runway,  located  200  miles  from  the  village.  The  airport 
is  heavily  guarded  by  the  local  military  to  keep  rebel  forces  out.  The  airport  flies  into  Australia, 
the  nearest  mainland  (750  miles  away),  and  from  there  can  connect  to  other  countries.  (See 
Map  3) 

Communication : 

Communication  on  the  island  and  off  the  island  is  possible,  but  is  limited  by  the  remoteness  of 
the  island.  The  Red  Cross  workers  have  working  mobile  phones,  however  the  batteries  died 
soon  after  they  contacted  the  American  Red  Cross  Headquarters.  The  church  where  the 
workers  are  trapped  does  not  have  a  telephone.  There  is  limited  radio  communication  on  the 
island  (3AM,  no  FM  stations).  There  are  no  television  stations  on  the  island.  Internet  access  on 
the  island  is  limited  to  government  workers  only. 

Condition  of  Red  Cross  Workers: 

The  workers  are  somewhat  safe  in  one  room  of  the  church.  They  have  no  source  of  water  except 
rainwater.  They  are  able  to  collect  food  easily  from  areas  around  them  but  that  supply  is 
limited  and  venturing  too  far  to  collect  food  is  dangerous.  Rescuers  should  be  aware  that 
workers  will  most  likely  be  dehydrated  and  suffering  from  malnutrition. 

One  of  the  workers  is  a  diabetic  in  need  of  insulin.  He  will  most  likely  go  into  insulin  shock  if 
not  treated  within  24  hours.  Another  worker  has  a  broken  leg.  The  third  worker  appears  to  be 
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healthy  at  this  time.  There  is  a  chance  that  workers  could  be  injured  from  the  outside  warfare 
and  that  their  location  may  not  be  safe  for  long.  They  must  be  rescued  within  24  hours. 


BACKGROUND  INFORMATION: 
Rebel  Forces 

X  The  Rebel  Forces  consists  of  500  trained  soldiers. 


X  They  guard  the  perimeters  of  their  captured  areas  at  ALL  times. 

X  They  have  no  night  vision  capability. 

X  The  have  Stinger  missiles.  (Stinger  missiles  are  hand-held,  infrared,  heat-seeking  rockets 
with  a  range  of  1  -  5  miles.) 

X  Their  small  arms  fire  consists  of  M-16  rifles  (range:  500yards)  and  9mm  pistols  (range: 
25  yards). 

X  They  have  land  mines.  (It  is  possible  that  local  roads  might  be  mined.) 

X  Their  weapons  storage,  communication  centers  and  anti-aircraft  locations  are  not  known 
to  the  U.S  or  Drapo  military. 

X  They  have  RPG’s  (rocket  propelled  grenades). 

X  Warfare  is  constant  between  rebels  and  local  military. 

X  They  have  easy  access  to  trucks  and  jeeps. 

X  They  are  aware  the  Red  Cross  workers  are  on  the  island,  but  unaware  of  their  exact 
location.  They  are  on-guard  for  possible  rescue  operations. 
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Navy  Seals  Assets 
Personnel: 

E  7-man  squad  consisting  of: 

~  l  Corpsman 

~  l  Radioman 

~  l  Heavy  Gunners 

~  2  Riflemen  with  M-i6  rifles 

~  2  Riflemen  with  M-i6  rifles  and  grenade  launchers 

E  Night  vision  capability 

E  Need  local  translator  if  communicating  with  villagers. 

E  Ability  to  be  virtually  undetected 

E  All  Navy  SEALS  are  trained  as  medics. 

E  Navy  SEALS  usually  initiate  covert  operations  from  the  sea.  (Covert  operations  are 
secret,  undetected  operations.) 

E  All  weapons  and  personnel  are  located  on  the  USS  Enterprise. 

E  The  USS  Enterprise  has  full  medical  facilities. 


Transportation  &  Weapons: 


E  Navy  Seahawk  (SH-6o):  A  twin-engine 
helicopter  used  for  anti-submarine 
warfare,  drug  interdiction,  cargo  lifts 
and  special  operations  in  the  day  or 
night  regardless  of  the  weather 
conditions. 

E  Navy  Hornet  (F-i8):  A  one  or  two  seat 
super  sonic  jet  used  for  air-to-air  or  air- 
to  ground  support. 

•  — - 

E  Zodiac:  A  7-man  inflatable  boat  which  can 
travel  at  speeds  up  to  15  miles/hour. 

JkaH  w 
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Army  Special  Forces  Assets 
Personnel: 

#  6-man  Special  Forces  Team  consisting  of: 

~  iTeam  Lead 
~  2  Snipers 
~  2  Team  Medics 

~  iTranslator (Speaks  several  languages,  but  NOT  Draponese.) 

^  Highly  trained  in  multiple  languages. 

^  Night  vision  capabilities 
^  Weapons:  M-16,  M-6o,  Grenades 

Army  Special  Forces  usually  initiate  covert  operations  from  the  land.  (Covert 
operations  are  secret,  undetected  operations.) 


Transportation  &  Weapons: 


#  Abram  Tanks:  Used  for  enemy 
suppression. 

_ _  . 

■ .  ■ 

&& -umm 

#  Blackhawk  Helicopter  (UH-6o):  A  twin 
engine  helicopter  used  for  special  ops  in 
the  day  or  night,  regardless  of  the  weather 
conditions. 

^  AH-64  Apache:  A  twin  engine  helicopter 
used  for  enemy  suppression. 

^  C-130  Hercules:  A  large  aircraft  used  for 
troop  and  cargo  transport,  paratrooper 
deployment  and  airborne  refueling. 
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Marine  Assets 
Personnel: 

►  12  Man  Team  that  includes: 

~  2  Medical  Corpsmen 
~  Use  of  M-16,  M-6o,  Grenades 
~  Night  vision  capabilities 

►  Marine  personnel  are  stationed  on  the  USS  Enterprise. 


Transportation  &  Weapons: 


►  AH-i:  An  attack  force  helicopter  used  to 
suppress  enemy  troops  and  provide  air 
support. 

^  -  - 

^  CH-53:  A  cargo/transport  helicopter  used 
in  amphibious  and  shore  operations. 
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Maps 


The  South  Pacific  Island  of  Drapo 
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Map  l:  Drapo  Island 


Scale 
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Map  2:  Drapo  Island 


Joral 
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Map  3:  North  Shore  of  Drapo  Island 
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Appendix  C:  News  Agency  Scenario  Primitives 


Scenario  l.  A  news  agency  deploying  a  reporter  and  support  assets  to  a  new  story 

1)  Identify  -  Operational  Constraints  -  time  -  environmental 

2)  Identify  -  Operation  Objectives  -  Operational  Policies 

3)  Mine  -  Information  Archives 

4)  Gather  -  Background  Information 

5)  Study  -  Operational  Environment 

6)  Identify  -  Information  Sources 

7)  Evaluate  -  Information  Source  -  reliability  -  accuracy 

8)  Identify  -  Information  Strategy 

9)  Identify  -  Transportation  Vehicle  -  requirements  -  type  -  Equipment  -  requirements  -  type  - 
Personnel  -  requirements  -  type 

10)  Identify  -  Transportation  Vehicle  -  location  -  status  -  Equipment  -  location  -  status  -  Personnel  - 
location  -  status 

11)  Identify  -  Personnel  -  capabilities 

12)  Identify  -  News  Story  -  location  -  accessibility 

13)  Dispatch  -  Transportation  Vehicle  -  type  -  operator  -  Personnel  -  numbers  -  type  -  Equipment  - 
type 

14)  Gather  -  Information 

15)  Analyze  -  Information  -  statistical  -  accuracy 

16)  Fact  Check  -  Information 

17)  Report  -  Information 

Scenario  2.  A  news  agency  assigning  a  reporter  to  independently  corroborate  a  news 
source 

1)  Identify  -  Operational  Constraints  -  time  -  environmental 

2)  Identify  -  Operation  Objectives  -  Operational  Policies 

3)  Identify  -  Personnel  -  capabilities 

4)  Dispatch  -  Personnel 

5)  Mine  -  Information  Archives 

6)  Identify  -  Information  Sources 

7)  Communicate  -  Contact 

8)  Interview  -  Contact 

9)  Evaluate  -  Information  Source  -  reliability  -  accuracy 

10)  Gather  -  Information 

11)  Analyze  -  Information  -  statistical  -  accuracy 

12)  Investigate  -  Information 

13)  Fact  Check  -  Information 

Scenario  3.  A  reporter  works  to  recruit  a  new  contact  for  a  story 

1)  Identify  -  Operational  Constraints  -  time  -  environmental 

2)  Identify  -  Operational  Objectives  -  Operational  Policies 

3)  Identify  -  Contact  -  location  -  type 

4)  Identify  -  Personnel  -  capabilities 

5)  Evaluate  -  Contact  -  availability  -  location 

6)  Communicate  -  Contact 

7)  Dispatch  -  Transportation  Vehicle  -  type  -  Personnel  -  type  -  Equipment  -  type 
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8)  Interview  -  Contact 

9)  Evaluate  -  Contact  -  reliability  -  accuracy 

Scenario  4.  A  news  agency  deploying  a  new  reporter  and  support  assets  to  follow-up  on 
an  existing  story 

1)  Identify  -  Operational  Constraints  -  time  -  environmental 

2)  Identify  -  Operation  Objectives  -  Operational  Policies 

3)  Mine  -  Information  Archives 

4)  Gather  -  Background  Information 

5)  Study  -  Operational  Environment 

6)  Gather  -  Existing  Information 

7)  Validate  -  Existing  Information 

8)  Identify  -  Information  Sources 

9)  Evaluate  -  Information  Source  -  reliability  -  accuracy 

10)  Identify  -  Information  Strategy 

11)  Identify  -  Transportation  Vehicle  -  requirements  -  type  -  Equipment  -  requirements  -  type  - 
Personnel  -  requirements  -  type  - 

12)  Identify  -  Transportation  Vehicle  -  location  -  status  -  Equipment  -  location  -  status  -  Personnel  - 
location  -  status 

13)  Identify  -  Personnel  -  capabilities 

14)  Identify  -  News  Story  -  location  -  accessibility 

15)  Dispatch  -  Transportation  Vehicle  -  type  -  Personnel  -  numbers  -  type  -  Equipment  -  type 

16)  Gather  -  Continuing  Information 

17)  Analyze  -  Information  -  statistical  -  accuracy 

18)  Fact  Check  -  Information 

19)  Integrate  -  Information  -  existing  -  continuing 

20)  Report  -  Information 

Scenario  5.  New  agency  to  synthesize  multiple  stories  to  create  a  new  edition  for  their 
customers 

1)  Identify  -  Operational  Constraints  -  time  -  spatial 

2)  Identify  -  Operational  Objectives  -  Operational  Policies 

3)  Identify  -  Personnel  -  requirements  -  Equipment  -  requirements 

4)  Dispatch  -  Personnel  type  -  Equipment  -  type 

5)  Identify  -  Information  Strategy 

6)  Gather  -  Information  -  new  -  background 

7)  Gather  -  Information  -  existing  -  continuing 

8)  Mine  -  Information  Archives 

9)  Validate  -  Existing  Information 

10)  Analyze  -  Information  -  statistical  -  accuracy 

11)  Fact  Check  -  Information 

12)  Integrate  -  Information 

13)  Report  -  Information 
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Appendix  D:  NEO  Scenario  Ontology  Diagrams 
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Figure  32:  NEO  Scenario  Noun  Primitives  from  Protege 


Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RTXX 

Report  No.  SERC-2010-RT-3 
May  31,  2010 


UNCLASSIFIED 

92 


'Organizational  Structure' 


UNCLASSIFIED 


Figure  33:  NEO  Scenario  Verb  Primitives  from  Protege 
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bdd  [SysML  Block  Definition]  High  Level  [1st  Level  links] 


NEO  Scenario 
Primitives 


7 


Equipment 

Interfaces 

Operations 

Personnel 

Risk  Assessment 

Infrastructure 

Transportation 

Environment 

notes 

«Definition» 
Equipment 
represents  the  tools 
and  hardware  that 
are  available  for 
use  in  carrying  out 
the  scenario. 

«Attributes» 

-hasType 

-hasLocation 

-hasStatus 

«Operations» 

-identifyEquipment 

-identifyType 

-identifyLocation 

notes 

«Definition» 

An  interface  is 
anywhere  that  two 
systems  or  people 
interact. 

«Attributes» 
-hasPointOfContact 
-haslnterfaced  Systems 
-hasInterfacedPeople 

«Operations» 

-interface 

-communicate 

-definelnterface 

notes 

«Definition» 
Operations  are 
the  operational 
parameters 
defining  how 
the  scenario 
should  be 
carried  out 
including 
Objectives, 
Operational 
Policies  and 
Operational 
Constraints. 

«Attributes» 

«Operations» 

notes 

«Definition» 

Personnel  are  objects  that  are  defined  as 
any  human  that  is  involved  in  the  scenario. 

«Attributes» 

-hasType 

-hasCapabilities 

-hasModesOfOperation 

-hasTraining 

-hasLocation 

-hasWeaknesses 

-hasReadinessLevel 

-hasStatus 

-hasLanguage 

-hasClassifi cation:  asset,  neutral,  - 
opposition,  unknown 
-has  Population 
-hasWeaponry 

«Operations» 

-deployPersonnel 

-gatherPersonnel 

-identifyPersonnelCapabilites 

-identifyPersonnelWeakenesses 

-trainPersonnel 

-contactPersonnel 

-identifyPersonnelStatus 

-identifyPersonnel  Readiness 

-locatePersonnel 

notes 

«Definition» 

Risk  Assessment  refers 
identification  of  risk  and 
its  components,  strategy 
for  mitigating  risk  and 
emergency  planning  . 

«Attributes» 

-hasRisk 

-hasRiskMitigationStrategy 

-hasContingencyPlan 

«Operations» 

-ri  skid  entifi  cation 

-ri  ski  m  p  act  A  sse  ssm  e  n  t 

-riskLi  kel  i  hood  Assessm  ent 

notes 

«Definition» 
Infrastructure  is  the 
basic  physical  and 
organizational 
structures  needed  for 
the  execution  of  the 
scenario. 

«Attributes» 

-hasSize 

-hasUsage 

-hasAge 

-hasStatus 

-hasLocation 

-hasReliability 

«Operations» 

-identifylnfrastrcutre 

-explorelnfrastructure 

-utilizelnfrastructure 

-exploitlnfrastructure 

-locatelnfrastructure 

-map  Infrastructure 

notes 
«Definition» 
Transportation  refers 
to  any  element  that 
facilitates  with  the 
movement  of 
personnel  and/or 
equipment. 

«Attributes» 

-hasStatus 

-hasSize 

-hasUsage 

-hasAge 

-hasReliability 

-hasLocation 

«Operations» 
-identifyT  ransportation 
-utilizeTransportation 
-movePersonnel 
-moveEquipment 

notes 
«Definition» 
Environment  refers  to 
the  location  of  the 
action  taken  in  the 
scenario  and  the 
location's  attributes. 
Environment  includes 
investigation  of 
physical,  economic, 
demographic,  political, 
support  and 
geographic  attributes 

«Attributes» 

«Operations» 

-studyOperationalEnvir 

onment 

Figure  35:  NEO  Scenario  Top  Level  Primitives  from  Sparx 
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bdd  [SysML  Block  Definition]  Equipment  [Equipment] 


NEO  Scenario  Primitives 

0 


Equipment 


notes 

«Defi  nition» 

Equipment  represents  the  tools 
and  hardware  that  are  available 
for  use  in  carrying  out  the 
scenario. 

«Attributes» 

-hasType 

-hasLocation 

-hasStatus 

«Operations» 

-identifyEquipment 

-identifyType 

-identifyLocation 

0 


Communications 

Equipment 

Medical  Equipment 

Personal  Equipment 

Weaponry 

notes 

notes 

notes 

notes 

«Defi  nition» 

<<Defi  niti  on» 

«Definition» 

«Definition» 

Medical  Equipment  refers 

Personal  Equipment  refers 

Weaponry  refers  to  any  equipment 

Communications 

to  any  equipment  that  is 

to  equipment  that  ischosen 

that  is  used  to  apply  force  for  the 

Equipment  refers  to  any 

used  to  provide  medical 

by  personnel  to  handle 

purpose  of  causing  harm  or  damage  to 

equipment  that  is  used 

services 

personal  up  keep  or  for 

persons,  animals  or  structures. 

to  transmit  voice  or  data 

other  personal  reasons, 

between  personnel 

«Attributes» 

including  hygiene, 

«Attributes» 

-hasMedicalSupportType 

mementos  and 

-hasCaliber 

«Attributes» 

entertainment  devices 

-hasAmmunitionType 

-hasFrequency 

«Operations» 

-hasRateOfFire 

-hasPowerSupply 

-equipMed  Equipment 

«Attributes» 

-hasUsage 

-useMedEquipment 

-hasUsageConstraints 

«Operations» 

-testMedEquipment 

«Operations» 

-hasRel  iabi  1  ity 

-equipComm  Equipment 

-equipPersonal  Equipment 

-use Comm  Equipment 

-usePersonal  Equipment 

«Operations» 

-testComm  Equipment 

-equipmWeaponry 

-use  Weaponry 

-testWeaponry 

Figure  36:  NEO  Scenario  Equipment  Primitives  from  Sparx 
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Figure  37:  NEO  Scenario  Interface  Primitives  from  Sparx 
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bdd  [SysML  Block  Definition]  Operations  [Operations] 


Objectives 


notes 

«Definition» 
Objectives  refer  to 
the  goals  in  order  to 
successfully  carry 
out  the  scenario. 

«Attributes» 
hasType:  Primary, 
Secondary 

«Operations» 

-identifyObjectives 

-completeObjectives 

-orderObjectives 


NEO  Scenario  Primitives 


1 


Operations 


notes 

«Definition» 

Operations  are  the  operational 
parameters  defining  how  the  scenario 
should  be  carried  out  including 
Objectives,  Operational  Policies  and 
Operational  Constraints. 

«Attributes» 


«Operations» 


T 


Operational  Policies 


notes 

«Definition» 

Operational  policies  refer  to 
those  written  statements  that 
communicate  an  organization's 
intent  in  order  to  provide 
guidance  to  personnel. 

«Attributes» 

-listsObjectives 
-hasRulesOf  Engagement 
-listsRequirements 
-li  stsResponsibi  I  ities 
-specifiesStandards 
-stateslntent 

«Operations» 


Operational  Constraints 


notes 

«Defi  nition» 

Operational  constraints  refer 
to  the  restrictions  an 
organization  and  its  personnel 
must  adhere  during  operation. 

«Attributes» 

-accessConstraints: 

-i  nfl  uence  Inform  ationStrategy: 
-timeConstraints: 
environmental  Constraints: 
-timeOfDayConstraints: 
-weatherConstraints: 

«Operations» 

-identifyConstraints: 


Figure  38:  NEO  Scenario  Operations  Primitives  from  Sparx 
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bdd  [SysML  Block  DefinitionIPersonnel  [Personnel] 


NEO  Scenario 
Primitives 


Personnel 


notes 

«Definition» 

Personnel  are  objects  that  are  defined  as  any  human 
that  is  involved  in  the  scenario. 

«Attributes» 

-hasType 

-hasCapabilities 

-hasModesOfOperation 

-hasTraining 

-hasLocation 

-hasWeaknesses 

-hasReadinessLevel 

-hasStatus 

-hasLanguage 

-hasClassification:  asset,  neutral,  -opposition,  unknown 

-has  Population 

-hasWeaponry 

«Operations» 

-deployPersonnel 

-gatherPersonnel 

-identifyPersonnelCapabilites 

-identifyPersonnelWeakenesses 

-trainPersonnel 

-contactPersonnel 

-identifyPersonnelStatus 

-identifyPersonnel  Readiness 

-locatePersonnel 


Organizational  Structure 


notes 

«Definition» 

Organizational  Structure  refers  to  a 
hierarchical  concept  of  entities  that 
collaborate  and  contribute  to  serve  one 
common  aim. 

«Attributes» 

-hasStructure 

-hasElements 

-hasLeadership 

«Operations» 
-identifyOrganizationalStructure 
-exploitOrganizational  Structure 


Figure  39:  NEO  Scenario  Personnel  Primitives  from  Sparx 
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Figure  40:  NEO  Scenario  Risk  Assessment  Primitives  from  Sparx 
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bdd  [SysML  Block  Definition]  Infrastructure  [Infrastructure] 


NEO  Scenario  Primitives 


Infrastructure 


notes 

«Definition» 

Infrastructure  is  the  basic  physical  and  organizational  structures 
needed  for  the  execution  of  the  scenario. 

«Attributes» 

-hasSize 

-hasUsage 

-hasAge 

-hasStatus 

-hasLocation 

-hasReliability 

«Operations» 

-identifylnfrastrcutre 
-explore  Infrastructure 
-utilizelnfra  structure 
-exploitlnfra  structure 
-locatelnfrastructure 
-maplnfrastructure 

o - 


Electrical 

Infrastructure 


notes 

«Definition» 
Electrical 
Infrastructure 
includes  power 
generation  plants 
and  power 
transmission  lines 

«Attributes» 

«Operations» 


Buildings 

Communication 

Infrastructure 

Land  Use 

Medical 

Infrastructure 

Waste  Mangement 
Infrastructure 

notes 

«Definition» 
Building  refers  to 
any  human-made 
structure  used  for 
supporting  or 
sheltering  any  use  or 
continuous 

occupancy. 

«Attributes» 

«Operations» 

notes 

<<Defi  ni  ti  on  » 
Land  use  refers  to 
human 

modification  of 
natural 

environment  into 
built 

environment. 

«Attributes» 

«Operations» 

notes 

«Definition» 

Communication 

Infrastructure  includes  any 
machinery  in  place  to 
facilitate  communication 
such  as  phone  lines, 
phone  booths  and  cellular 
infrastructure. 

«Attributes» 

«Operations» 

notes 

«Definition» 
Medical 
Infrastructure 
refers  to  facilities 
that  provide 
medical  services 

«Attributes» 

«Operations» 

notes 

«Definition» 

Sewage 

Infrastructure  refers 
to  the  facilities  that 
transport,  process 
and  dispose  of 
waste 

«Attributes» 

«Operations» 

Figure  41:  NEO  Scenario  Infrastructure  Primitives  from  Sparx 
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bdd  [SysML  Block  Definition]  Transportation  [Transportation] 


Transportation 


notes 

«Definition» 

Transportation  refers  to  any 
element  that  facilitates  with  the 
movement  of  personnel  and/or 
equipment. 


«Attributes» 

-hasStatus 

-hasSize 

-hasUsage 

-hasAge 

-hasReliability 

-hasLocation 


«Operations» 
-identifyT  ransportation 
-uti  I  i  zeT  ran  spo  rtati  on 
-movePersonnel 
-moveEquipment 


Vehicles 

Vehicle  Constraints 

notes 

notes 

«Definition» 

«Definitions» 

Vehicles  are  objects  that  are  used  to  transport 

Vehicle  constraints  are 

personnel  and/or  equipment.  In  this  context 

constrai  nts  prohi  bitti  ng 

they  include  both  mechanized  and  non 

or  impairing  the 

mechanized  transportation  objects. 

«Attributes» 

operation  of  a  specific 
vehicle. 

-hasType 

«Attributes» 

-hasConstraints 

-haslmpact 

-hasWeaponry 

-hasMaxSpeed 

-hasLocation 

-hasM  ax  Weight 

-hasEquipment 

-hasMaxDepth 

-hasOperators 

-hasMaxAltitude 

-hasUsage 

-hasRange 

-hasFuelType 

-hasCargoCapacity 

-hasFuelCapacity 

-hasPersonnelCapacity 

-hasFuelQuantity 

-h  asOpe  rato  rCapaci  ty 

-hasRefuel  Method 

-hasSoundProfile 

-hasT ravel  Speed 
-hasT ravel  Route 

-hasWeatherConstraints 

-hasT ravel  Distance 

«Operations» 

-hasT  ravelTime 

-identifyConstraints 

-hasT  ravelDirection 

«Operations» 

-move  Direction 

-refuelVehicle 

-transportPersonnel 

-transportEquipment 

-dispatchVehicle 

-towVehicle 

-towEquipment 

-analyzeConstraints 

Transportation  Infrastructure 


notes 

«Definition» 

Transportation  Infrastructure  refers  to  fixed 
installations  that  allow  a  vehicle  to  operate. 

«Attributes» 

-hasRoads 

-hasWaterways 

-hasPorts 

-hasSeaAccess 

-hasAirAccess 

-hasAirports 

-hasAirLanes 

-hasShippingLanes 

-hasConstraints 

-hasReliability 

-hasAccessibility 

-hasPublicTransport 

«Operations» 

-maplnfrastructure 
-anal  yze  I  nf  rrastructu  re 
-uselnfrastructure 
-exploitlnfrastructure 


Figure  42:  NEO  Scenario  Transportation  Primitives  from  Sparx 
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bdd  [SysML  Block  Definition]  Environment  [Environment]  ▼ 

NEO  Scenario  Primitives 


Environment 


notes 

«Defi  nition» 

Environment  refers  to  the  location  of  the  action  taken  in  the  scenario  and  the 
location's  attributes.  Environment  includes  investigation  of  physical, 
economic,  demographic,  political,  support  and  geographic  attributes 

«Attributes» 


«Operations» 
-studyOperational  Environment 


Geographic  Environment 


notes 

«Defi  nition» 

Geographic  Environment 
refers  to  the  geography 
associated  with  the 
scenario,  including 
geographic  history  and 
neighboring  countries 

«Attributes» 

-hasNeighbors 

-hasHistory 

-hasArea 

-hasBorders 

«Operations» 

-collectGeographyMaps 

-analyzeGeographyMaps 

-defineAreaOf  Interest 

-collectGeographicHistory 

-analyzeGeographicHistory 

-exploitGeography 


Demography 


notes 

«Definition» 
Demography  refers  to 
the  study  of  human 
populations 

«Attributes» 

-hasPopulation 

-hasPopulationDensity 

-hasEducation 

-hasReligion 

-hasEthnicity 

-hasBirthRate 

-hasDeathRate 

-hasGenderMakeup 

«Operations» 
-research  Demography 
-analyzeDemography 
-exploitDemography 


Economic  Environment 


notes 
«Defi  nition» 

Economic  Environment 
refers  to  the  field  of  study 
involving  the  production, 
distribution,  and 
consumption  of  goods 
and  services 

«Attributes» 

-hasExports 

-haslmports 

-hasHistory 

-hasGDP 

-hasEconopmicindicators 

-hasCurrency 

-hasServices 

-hasGoods 

«Operations» 

-researchEconomics 

-analyzeEconomics 

-exploitEconomics 


Support  Environment 


notes 
«Defi  nition» 

Support  Environment 
refers  to  the  assets  an 
organization  or  nation 
has  within  an  area  of 
interest  in  the  scenario 

«Attributes» 

-hasObjectiveRallyPoint 

-hasRegionalAssets 

-hasReadiness 

-hasStatus 

-hasPopulation 

-hasCapabilities 

«Operations» 

-identifySupportEnviron 

-analyzeSupportEnviron 

-mapSupportEnviron 


Political  Environment 


notes 

«Definition» 

Political  Environment 
refers  to  the  theory  of 
politics  and  analysis  of 
political  systems  and 
behavior 

«Attributes» 
-hasGovernmentSystem 
-hasHistory 
-hasPolitical  Parties 

«Operations» 
-identifyPoiitical  Allies 
-identifyPoliticalOpposition 
-research  Political  Environ 
-analyzePolitical  Environ 
-monitorPoliti cal  Environ 
-exploitPolitical  Environ 


Physical  Environment 


notes 

«Definition» 

Physical  Environment  refers  to  all 
living  and  non-living  things  in  the 
area  of  interest  of  the  scenario. 

«Attributes» 

-hasSpecies 

-hasCurrentWeather 

-hasWeatherForecast 

-hasTopography 

-hasFood  Sources 

-hasDangerousSpecies 

«Operations» 

-mapTopography 

-analyzeT  opography 

-collectWeatherForecast 

-analyzeWeatherForecast 

-mapWeatherForecast 

-catalogueSpecies 

-identifyDangerousSpecies 

-identifyFoodSources 

-mapBodiesOfWater 


Figure  43:  NEO  Scenario  Environment  Primitives  from  Sparx 
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